Document processing system

ABSTRACT

A document processing system is disclosed that is capable of processing both fixed-format and unfixed-format hand written paper documents. The document processing system includes an encoding unit that encodes a sheet ID for identifying a hand written first document on a sheet to generate a coded sheet ID; a decoding unit that decodes the coded sheet ID; a document-sheet ID association unit that associates the sheet ID with a document ID assigned to a computerized second document; a printing unit that acquires the sheet ID and prints the coded sheet ID on the first document; a sheet ID management unit that manages the sheet ID; an information acquisition unit that acquires the sheet ID decoded by the decoding unit, and hand-written data from the first document on which the coded sheet ID is printed; and a process-sheet ID association unit that associates the sheet ID with a process ID of a process for processing the hand-written data acquired by the information acquisition unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a document processing system forprocessing a hand written document on a sheet, and a method thereof.

2. Description of the Related Art

Along with the progress in computerization of office work, documents inelectronic form (referred to as “electronic document” below whereappropriate) are more and more widely used. However, because quality ofcharacters displayed by a CRT (Cathode-Ray Tube) or a LCD (LiquidCrystal Display) is not as good as that printed on paper in manyaspects, people usually print the electronic documents on paper for use(“documents printed on paper” is referred to as “paper document” belowwhere appropriate).

For example, Japanese Patent Gazette No. 3219251 (referred to as“reference 1” hereinafter) discloses a document processing device fordocument creation and editing, which enables a number of authors andeditors to easily access and edit original electronic documents via thepaper documents obtained by printing these electronic documents, andenables acquiring the original electronic documents from the paperdocuments and exchanging of information of the original electronicdocuments and the paper documents.

Japanese Laid-Open Patent Application No. 10-63820 (referred to as“reference 2” hereinafter) discloses an apparatus and a method thatmakes original electronic documents in connection with the paperdocuments thereof, provides means for determining an original electronicdocument from the corresponding paper document and editing the originalelectronic document, and reflects the editing of the original electronicdocument on the paper document to make the original electronic documentand the paper document are identical.

In Japanese Laid-Open Patent Application No. 2002-222075 (referred to as“reference 3” hereinafter), “bar code transformation means” and “firstprinting means” are disclosed functioning as means for assigningidentifiers (ID) of documents and creating a form (paper document), a“management table” is disclosed to manage connections betweenidentifiers (ID) of documents and electronic documents, a “printingsheet” is disclosed as means for acquiring paper ID and writinginformation, an “optical reading device” is disclosed as stroke-typeinput means, “bar-code decoding means” is disclosed as means foracquiring a paper ID from an image where a bar code is used as anidentifier, and “additional edition reading processing means” isdisclosed as means for interpreting a layout from an image of amanuscript with additional editing and performing appropriate processingin different regions.

In Japanese Laid-Open Patent Application No. 2002-312478 (referred to as“reference 4” hereinafter), means for assigning identifiers (ID) ofdocuments and for creating a form (paper document) is disclosed indetail; the connection between electronic documents and identifiers (ID)on paper documents is illustrated in FIG. 15 of reference 4 with thedescription “search a format file data base via a document ID”, and isdescribed in detail in paragraphs No. 0056 and No. 0057; means foracquiring a paper ID and written information is illustrated in FIG. 26,FIG. 2, and FIG. 17 as a scanner, a pen-type panel, and a pad-typepanel, and stroke-type input means is also disclosed in these figures.

Further, in reference 4, a code reader 15 is illustrated in FIG. 19 asmeans for acquiring a paper ID, and an identifier such as a bar code isdescribed in paragraphs No. 0034 through No. 0036; means forinterpreting a layout from an image of a manuscript with additionalediting and performing appropriate processing in different regions isdescribed by successive processing steps in FIG. 15. For example, thereare descriptions like “clustering writing coordinates according toformat information”, “processing the clustered coordinate groupsaccording to processing descriptions of format information (OCR, Imageprocessing)”, “registering data obtained by OCR processing with a recorddatabase”, “saving the database obtained by Image processing to .shc and.tki files as coordinate data sequences”; and means for acquiring apaper ID and a document ID, and processing information is described inparagraphs No. 0077 and No. 0078.

Japanese Laid-Open Patent Application No. 2002-281179 (referred to as“reference 5” hereinafter) discloses a method in which a device havingcommunication functions and equipped with an optical reading device isused to read a bar code including an operation identifier and an objectidentifier, thereby providing services for customs according to the barcode information.

Japanese Laid-Open Patent Application No. 2002-183520 (referred to as“reference 6” hereinafter) discloses a method involving a check box forselection of commodity and selection of processing methods and textwriting regions, and enabling providing services for customs by using abar code obtained by coding a printed matter ID, an arrangement of thecheck box, and position information of the text writing regions, andenabling providing services desired by users by using printed matter andcopier-facsimile complexes, which are familiar to the users.

Japanese Laid-Open Patent Application No. 2002-215768 (referred to as“reference 7” hereinafter) relates to a technique for providing servicesfor customs, which involves distributing to custom paper provided withcodes including information of addresses for communication destination,information indicating coordinates on the paper, and information oftypes of the paper, and thereby enabling the custom to write on thepaper with the paper held in the hands of the custom. In addition, inreference 7, an input device is disclosed that is able to read the codesprovided on the paper, and transmits the address information in thecodes to the communication destination. This input device transmits theobtained information to a server, the server obtains the coordinateinformation and media identification information in the codes, andperforms appropriate operations in response to the obtained information.

In the aforesaid reference 6, the custom need to read the printedmatter, on which the additional editing is written, by acopier-facsimile complex. In contrast, in reference 7, it is the inputdevice, such as writing materials, that reads and transmits the codesincluding various kinds of information.

Japanese Laid-Open Patent Application No. 11-96166 (referred to as“reference 8” hereinafter) discloses a document information managementsystem that searches for and outputs document files from a databasestoring a number of electronic documents. Specifically, the documentinformation management system uses a recording medium, which has adocument information area for storing document information and a barcode information area for storing bar code information that is obtainedby converting electric information corresponding to the documentinformation, and a marking unit for selecting desired words from therecording medium and assigning a selection mark. Based on the selectionmark and the bar code information, the document information managementsystem searches the database and outputs the appropriate document files.

In reference 1, because editing is after all performed on the paperdocument, a user has to use a computer to call up the electronicdocument corresponding to the given paper document, and revises andedits contents of the document on the electric form document. There isnot any problem in doing this work if the user is in his office.However, if the user is at his business travel destination or in atransportation facility, and even if the user desires to revise or editdocuments in his spare time, usually the user cannot do that becauseusually the user does not carry with him an editing device able toefficiently and electrically edit documents.

For example, because a notebook personal computer has low portabilitydue to large size and weight, and has low practical operability due tothe short duration of the batteries, usually, one does not tend to carrythe notebook personal computer with him in business travel. This is truealso for a compact PDA (Personal Digital Assistant) which has a smalldisplay unit and an input device having poor input functions.

The apparatus or the method disclosed in reference 2 merely allows theadditional editing made on a paper document to be inserted into anelectronic document. For example, by simply inserting an additionalimage into the electronic document, the apparatus does nothing buteliminate differences between the electronic document and the paperdocument and create an electronic document having the same appearance asthe paper document. Thus, the electronic document addressed in reference2 is no more than a replacement of the paper document. Therefore, thecreated electronic document has not any practical use except being read.

For example, reference 2 allows one to read items filled out on aquestionnaire, but one has to collect the questionnaires by himself,just like the conventional paper documents. This work is troublesome,and automatic collection is required.

In addition, automatic processing of additional editing is not limitedto collecting questionnaires, but is usable in many applications. Thetechnique disclosed in reference 2 cannot realize automatic processingof additional editing.

In the reference 3, the original electronic document is not connectedwith the additional editing, and the method disclosed in the reference 3is neither configured to perform registration of a processing ID orother processing information, nor to edit the form itself, nor toacquire a paper ID, a document ID, or other processing information.

In reference 4, it is described that it is sufficient to make theoriginal electronic document and patient information be connected witheach other, but connection between the original electronic document andthe additional editing is not addressed. In addition, the methoddisclosed in reference 4 is not configured to perform registration of aprocessing ID or other processing information, or to edit the formitself.

By the method disclosed in the reference 5, only services correspondingto the prepared bar code information can be provided, that is, the rangeof user's selection is quite limited.

The method disclosed in the reference 6 is capable of processing adocument having a fixed format (referred to as “fixed-format documentprocessing” below), but not capable of processing a document without afixed format (referred to as “unfixed-format document processing”below). This is also true for the method disclosed in reference 7.

The method disclosed in the reference 8 allows direct access to anelectronic document even from a paper document not in a hypertext form.This method is capable of fixed-format document processing specifiedbeforehand, but is not capable of other processing. In addition, thismethod is specialized to search for document files in a database, butcannot be used for other applications.

SUMMARY OF THE INVENTION

It is a general object of the present invention to solve one or moreproblems of the related art.

A specific object of the present invention is to provide a documentprocessing system capable of processing both fixed-format andunfixed-format hand written paper documents, and a method and a programthereof.

According to a first aspect of the present invention, there is provideda document processing system for processing a hand-written firstdocument on a sheet. The document processing system includes an encodingunit configured to encode a sheet ID for identifying the first documentto generate a coded sheet ID; a decoding unit configured to decode thecoded sheet ID; a document-sheet ID association unit configured toassociate the sheet ID with a document ID assigned to a computerizedsecond document; a printing unit configured to acquire the sheet ID andprint the coded sheet ID on the first document; a sheet ID managementunit configured to manage the sheet ID; an information acquisition unitconfigured to acquire the sheet ID decoded by the decoding unit, andhand-written data from the first document on which the coded sheet ID isprinted; and a process-sheet ID association unit configured to associatethe sheet ID with a process ID of a process for processing thehand-written data acquired by the information acquisition unit.

According to a second aspect of the present invention, there is provideda document processing system for processing a hand-written firstdocument on a sheet. The document processing system includes an encodingunit configured to encode a sheet ID for identifying the first documentto generate a coded sheet ID; a decoding unit configured to decode thecoded sheet ID; a document-sheet ID association unit configured toassociate the sheet ID with a document ID assigned to a computerizedsecond document; a printing unit configured to acquire the sheet ID andprint the coded sheet ID on the first document; a sheet ID managementunit configured to manage the sheet ID; an information acquisition unitconfigured to acquire the sheet ID decoded by the decoding unit andhand-written data from the first document on which the coded sheet ID isprinted; a process-sheet ID association unit configured to associate thesheet ID with a process ID of a process for processing the hand-writtendata acquired by the information acquisition unit; and an ID conversionunit configured to convert the sheet ID to the document ID and theprocess ID.

According to a third aspect of the present invention, there is provideda document processing system for processing a hand-written firstdocument on a sheet. The document processing system includes an encodingunit configured to encode a sheet ID for identifying the first documentto generate a coded sheet ID; a decoding unit configured to decode thecoded sheet ID; a printing unit configured to acquire the sheet ID andprint the coded sheet ID on the first document; a document-sheet IDassociation unit configured to associate the sheet ID with a document IDassigned to a second document, said second document being a computerizeddocument having a predetermined format; a sheet ID management unitconfigured to manage the sheet ID; an information acquisition unitconfigured to acquire the sheet ID decoded by the decoding unit and thehand-written data from the first document on which the coded sheet ID isprinted; a process-sheet ID association unit configured to associate thesheet ID with a process ID of a process for processing the hand-writtendata acquired by the information acquisition unit; a process IDmanagement unit configured to manage the process ID; and a decompositionstorage unit configured to decompose the hand-written data acquired bythe information acquisition unit based on layout information of thepredetermined format and the process ID, and store the decomposedhand-written data.

As an embodiment, the printing unit comprises a printing service unitthat prints the first document based on the second document. Theprinting service unit includes a first unit that acquires the sheet IDassigned to the first document; a second unit that sends the acquiredsheet ID to the encoding unit to generate the coded sheet ID; and athird unit that superposes the coded sheet ID on the second document.

As an embodiment, the printing unit includes a first unit that acquiresthe sheet ID assigned to the first document; a second unit that sendsthe acquired sheet ID to the encoding unit to generate the coded sheetID; a third unit that superposes the coded sheet ID on the seconddocument; and a printing service unit that outputs the second documentsuperposed with the coded sheet ID for printing the first document.

As an embodiment, the printing unit comprises a printing service unitthat prints the first document based on a printing image. The printingservice unit includes a document conversion unit that converts thesecond document to the printing image; a first unit that acquires thesheet ID assigned to the first document; a second unit that sends theacquired sheet ID to the encoding unit to generate the coded sheet ID;and a third unit that superposes the coded sheet ID on the printingimage.

As an embodiment, the printing unit comprises a document conversion unitthat converts the second document to a printing image or a pagedescription language sequence; and a printing service unit that printsthe first document based on the printing image or the page descriptionlanguage sequence. The printing service unit includes a first unit thatacquires the sheet ID assigned to the first document; a second unit thatsends the acquired sheet ID to the encoding unit to generate the codedsheet ID; and a third unit that superposes the coded sheet ID on theprinting image.

As an embodiment, the sheet ID management unit stores document-sheet IDassociation information in a non-volatile storage unit, thedocument-sheet ID association information is used for associating thesecond document with the sheet ID.

As an embodiment, the document-sheet ID association information isaccessible from a communication network.

As an embodiment, the sheet ID management unit associates the seconddocument with the sheet ID in response to a request transmitted from acommunication network.

As an embodiment, the sheet ID management unit associates a documenttitle of the second document with the sheet ID in management.

As an embodiment, the sheet ID management unit stores the seconddocument associated with a sheet ID.

As an embodiment, the information acquisition unit includes an imagepick-up unit that picks up an image of an object including the firstdocument; a driving unit that drives the image pick-up unit relative tothe first document; a control unit that controls the driving unit todivide the first document into a plurality of regions for imagepicking-up; and a combination unit that combines images of the dividedregions of the first document into one image.

As an embodiment, the information acquisition unit comprises a sheet IDimage pick-up unit that picks up an image of the coded sheet ID.

As an embodiment, the information acquisition unit comprises an imagepick-up unit that picks up an image of an object including the firstdocument; a driving unit that drives the image pick-up unit relative tothe first document; a detection unit that detects a position of thecoded sheet ID on the object; and a control unit that determines aposition of the image pick-up unit so that the image pick-up unit isable to pick up an image of the coded sheet ID, and determines amagnification to be applied for picking up an image of the coded sheetID. The image obtained by the image pick-up unit is associated with thesheet ID obtained from the image of the coded sheet ID.

As an embodiment, the information acquisition unit comprises an imagepick-up unit that picks up an image of an object including the firstdocument; and a detection unit that detects a position of the codedsheet ID on the object. The sheet ID obtained from an image of the codedsheet ID taken by the image pick-up unit is associated with the documentID and the process ID by the ID conversion unit.

As an embodiment, the information acquisition unit comprises an imagepick-up unit that picks up an image of an object including the firstdocument; and a sheet ID display unit provided in the object fordisplaying the coded sheet ID, said sheet ID display unit being storableand including the coded sheet ID.

As an embodiment, the information acquisition unit comprises an imagepick-up unit that picks up an image of an object including the firstdocument; and an optical projecting unit that optically projects thecoded sheet ID so as to allow the image pick-up unit to pick up an imageof the coded sheet ID.

As an embodiment, the information acquisition unit comprises an imagepick-up unit that picks up an image of an object including the firstdocument; and a display unit provided in the object for displaying animage of the coded sheet ID taken by the image pick-up unit.

As an embodiment, the process ID management unit manages program-processID association information that associates the process ID with a programfor executing the process for processing the hand-written data.

As an embodiment, the process ID management unit stores theprogram-process ID association information in a non-volatile storageunit.

As an embodiment, the program-process ID association information isstored in a database accessible from a communication network.

As an embodiment, the program-process ID association informationincludes information of a position of storing data to be processed bythe program.

As an embodiment, the program-process ID association informationincludes information of a position of a service to be provided byexecuting the program.

As an embodiment, the process ID management unit associates the processID with the program in response to a request transmitted from acommunication network.

According to a fourth aspect of the present invention, there is provideda document processing method for processing a hand-written firstdocument on a sheet. The document processing method includes the stepsof associating a sheet ID with a process ID, the sheet ID being used foridentifying the first document, the process ID being used for indicatinga process for processing hand-written data on the first document;associating the sheet ID with a document ID assigned to a computerizedsecond document; assigning the sheet ID to the first document; encodingthe sheet ID to generate a coded sheet ID; and acquiring the coded sheetID and the hand-written data from the first document, the coded sheet IDbeing printed on the first document.

According to a fifth aspect of the present invention, there is provideda document processing method for processing a hand-written firstdocument on a sheet. The document processing method includes the stepsof associating a sheet ID with a process ID, the sheet ID being used foridentifying the first document, the process ID being used for indicatinga process for processing hand-written data on the first document;associating the sheet ID with a document ID assigned to a computerizedsecond document; associating the process ID with a program for executingthe process for processing the hand-written data; assigning the sheet IDto the first document; encoding the sheet ID to generate a coded sheetID; printing the first document with the coded sheet ID thereon;acquiring the coded sheet ID and the hand-written data from the firstdocument, the coded sheet ID being printed on the first document; andconverting the sheet ID to the document ID and the process ID.

According to a sixth aspect of the present invention, there is provideda document processing method for processing a hand-written firstdocument on a sheet. The document processing method includes the stepsof associating a sheet ID with a process ID, the sheet ID being used foridentifying the first document, the process ID being used for indicatinga process for processing hand-written data on the first document;associating the sheet ID with a document ID assigned to a computerizedsecond document having a predetermined format; assigning the sheet ID tothe first document; encoding the sheet ID to generate a coded sheet ID;and acquiring the coded sheet ID and the hand-written data from thefirst document, the coded sheet ID being printed on the first document;and decomposing the hand-written data based on layout information of thepredetermined format and the process ID and storing the decomposedhand-written data.

According to a seventh aspect of the present invention, there isprovided a program executable in a computer for processing ahand-written first document on a sheet. The program makes the computerexecute the steps of associating a sheet ID with a process ID, the sheetID being used for identifying the first document, the process ID beingused for indicating a process for processing hand-written data on thefirst document; associating the sheet ID with a document ID assigned toa computerized second document; assigning the sheet ID to the firstdocument; encoding the sheet ID to generate a coded sheet ID; andacquiring the coded sheet ID and the hand-written data from the firstdocument, the coded sheet ID being printed on the first document.

According to an eighth aspect of the present invention, there isprovided a program executable in a computer for processing ahand-written first document on a sheet. The program makes the computerexecute the steps of associating a sheet ID with a process ID, the sheetID being used for identifying the first document, the process ID beingused for indicating a process for processing hand-written data on thefirst document; associating the sheet ID with a document ID assigned toa computerized second document; associating the process ID with aprogram for executing the process for processing the hand-written data;assigning the sheet ID to the first document; encoding the sheet ID togenerate a coded sheet ID; printing the first document with the codedsheet ID thereon; acquiring the coded sheet ID and the hand-written datafrom the first document, the coded sheet ID being printed on the firstdocument; and converting the sheet ID to the document ID and the processID.

According to a ninth aspect of the present invention, there is provideda program executable in a computer for processing a hand-written firstdocument on a sheet. The program makes the computer execute the steps ofassociating a sheet ID with a process ID, the sheet ID being used foridentifying the first document, the process ID being used for indicatinga process for processing hand-written data on the first document;associating the sheet ID with a document ID assigned to a computerizedsecond document having a predetermined format; assigning the sheet ID tothe first document; encoding the sheet ID to generate a coded sheet ID;and acquiring the coded sheet ID and the hand-written data from thefirst document, the coded sheet ID being printed on the first document;and decomposing the hand-written data based on layout information of thepredetermined format and the process ID and storing the decomposedhand-written data.

According to a 10th aspect of the present invention, there is provided adocument processing system for processing a hand-written first documenton a sheet. The document processing system comprises a document-sheet IDassociation unit configured to associate a sheet ID for identifying thefirst document with a document ID assigned to a computerized seconddocument; and a sheet ID management unit configured to manage the sheetID based on document-sheet ID association information, which is used forassociating the second document with the sheet ID.

According to the present invention, it is possible to provide a documentprocessing system capable of processing both fixed-format andunfixed-format hand written documents on a sheet, and provide a methodand a program of the system.

These and other objects, features, and advantages of the presentinvention will become more apparent from the following detaileddescription of preferred embodiments given with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view showing an overall configuration of adocument processing system of an embodiment according to the presentinvention;

FIG. 2 is a block diagram showing a hardware configuration of thecomputer 101 or 103 in FIG. 1;

FIG. 3 is a block diagram showing functional constituent elements Athrough K of the document processing system according to the embodimentof the present invention;

FIG. 4 is a flowchart showing a first embodiment of the element A 121;

FIG. 5 is a flowchart showing a second embodiment of the element A 121;

FIG. 6 is a flowchart showing a third embodiment of the element A 121;

FIG. 7 is a flowchart showing a fourth embodiment of the element A 121;

FIG. 8 is a flowchart showing a fifth embodiment of the element A 121;

FIG. 9 is a view showing an example of a management table used in theelement B 122 for associating a paper ID with a document ID;

FIG. 10 is a view showing another example of the management table usedin the element B 122 for associating a paper ID with a document ID;

FIGS. 11A and 11B are views showing an example of a paper document;

FIG. 12 is a view showing another example of the paper document;

FIG. 13 is a perspective view of a configuration of the image capturingdevice according to the embodiment of the present invention;

FIG. 14 is a block diagram showing configurations of the image pick-upunit 201, the driving unit 202, and the control unit 205;

FIG. 15 is a diagram showing a configuration of the image pick-up unit201;

FIG. 16 is a view of an example of a paper document;

FIG. 17 is a flowchart showing operations of the image capturing device;

FIG. 18 is a schematic view showing movement of the image pick-up unit201;

FIG. 19 is a view for schematically showing divisional images;

FIG. 20 is a view for schematically showing combination of thedivisional images;

FIG. 21 is a view showing a corrected image;

FIG. 22 is a perspective view showing an example of the image capturingdevice;

FIG. 23 is a perspective view showing another example of the imagecapturing device;

FIG. 24 is a perspective view showing another example of the imagecapturing device;

FIG. 25 is a perspective view showing another example of the imagecapturing device;

FIG. 26 is a perspective view showing another example of the imagecapturing device;

FIG. 27 is a perspective view showing another example of the imagecapturing device;

FIG. 28 is a block diagram showing a written information input device60;

FIG. 29 is a perspective view showing an appearance of the writteninformation input device 60;

FIG. 30 is a flowchart showing operations of a portable digitizer as animplement of the written information input device 60;

FIG. 31 is a table showing data stored and transmitted by the writteninformation input device 60;

FIG. 32 is a block diagram showing an image input device 88;

FIG. 33 is flowchart showing the operation of the image input device 88;

FIG. 34 is a flowchart showing the image erecting processing;

FIG. 35 is a flowchart showing the region extraction processing;

FIG. 36 is a diagram for schematically showing the operation of labelingthe black pixels;

FIG. 37 is a diagram for schematically showing the operation ofdetecting coordinates of the circumscribing rectangle enclosing theblack pixels having the same label;

FIG. 38 is a diagram for schematically showing the operation ofcombining the circumscribed rectangles;

FIG. 39 is a block diagram showing a configuration of a bar-code readingdevice as an embodiment of the element E 125;

FIG. 40 is a view showing the operation of the vertex candidatedetection unit 11 for detecting the candidates of the vertices of aninput image;

FIG. 41 is an enlarged view of a portion of the FIG. 40 for showing theoperation of determining whether the black pixels A, B, C and D arecandidates of vertices of the input image;

FIG. 42 is a view showing the operation of the code frame detection unit12 for detecting a code frame of an input image;

FIG. 43 is a flowchart showing the operation of reading thetwo-dimensional code;

FIG. 44 is a view for schematically showing an example of reading animage having trapezium cells or having cells of various sizes by anoptical system capable of obliquely picking up an image;

FIG. 45 is a view of an example of a two-dimensional code obtained bytransformation, in which the boundaries of cells are stepwise;

FIG. 46 is a view of another example of a two-dimensional code obtainedby transformation, in which the boundaries of cells are stepwise;

FIG. 47 is a view showing of the projecting transformation on atwo-dimensional code;

FIG. 48 is diagram showing the projecting transformation fortransforming an object two-dimensional code 67 to an electricallycreated two-dimensional code 68;

FIG. 49 shows equations (5) and (6) for calculating transformationcoefficients;

FIG. 50 is a block diagram showing a configuration of a two-dimensionalcode creating device;

FIG. 51 is view for schematically showing data arrangement in thetwo-dimensional code creation unit 23;

FIG. 52 is flow chart showing the operation of creating thetwo-dimensional code;

FIG. 53 is a view for schematically showing data arrangement;

FIG. 54 is a view showing an example of a management table;

FIG. 55 is a view of an example of a paper document;

FIG. 56 is a view of an information table;

FIG. 57 is a view showing a form structure entity;

FIG. 58 is view for schematically showing an interface of the formcreation program;

FIG. 59 is view for schematically showing creation of a field forinputting a name by using the text box 152;

FIG. 60 is a view of a table including groups of properties of contents;

FIG. 61 is a view for schematically showing decomposition ofhand-written information into data;

FIG. 62 is a flowchart showing the operation of decomposing hand-writteninformation written down on a form into data;

FIG. 63 is block diagram of the relation between the element K 131 withother elements; and

FIG. 64 shows equations (1) through (4) for calculating transformationcoefficients.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Below, preferred embodiments of the present invention are explained withreference to the accompanying drawings.

A general description is made before explanation of an overallconfiguration of an embodiment of the present invention.

The document processing system according to an embodiment of the presentinvention connects documents on paper with documents in digital form.For example, in a conventional processing system, one opens a form on ascreen from a Web site, types on the keyboard of his computer to inputdata to fill out the form, and strokes a “Submit” key to submit thefilled-out form to a server. Then, the system processes, for example,for shopping.

Similarly, in the document processing system of the embodiment of thepresent invention, one can fill out a form on paper with a pen or otherwriting tools, and send the hand-written information filled out in theform to a server for the same subsequent processing as in theconventional system. In other words, the document processing system ofthe embodiment of the present invention enables hand-written informationon paper to be processed in the same way as digital data input by akeyboard.

FIG. 1 is a schematic view showing an overall configuration of thedocument processing system of an embodiment according to the presentinvention.

In FIG. 1, the document processing system includes a computer 101, forexample, a personal computer, for preparing various forms, a printer102, a computer 103, for example, a personal computer, for processingforms, a digital PAD 104, a scanner 105, a data server 106, and acommunication network 107.

The computer 101 prepares paper forms, which are filled out by handwriting. The printer 102 prints the paper forms. Following the format ofthe paper form, the computer 103 processes the hand-written informationon a filled-out paper form. The scanner 105 reads the hand-written paperform. The digital PAD 104 successively obtains coordinates of handwriting, and resultantly obtains information filled out on the paperform. The server 106 has a data base used when processing the filled outpaper forms.

In the following description, the images of the forms produced by thescanner 105, and the hand writing coordinates obtained by the digitalPAD 104 (that is, the stroke data) are generically referred to as “handwritten information”. In addition, in the following, when necessary, thecomputer 101 and the printer 102 are collectively referred to as “formpreparation device 108”, the computer 103, the scanner 105, and thedigital PDA 104 are collectively referred to as “form processing device109”, the data server 106 is referred to as “storage device 100”.

FIG. 2 is a block diagram showing a hardware configuration of thecomputer 101 or 103 in FIG. 1.

As illustrated in FIG. 2, the computer includes an input device 31, adisplay unit 32, a drive device 33, a storage medium 34, a secondarystorage device 35, a memory device 36, a processor 37, and an interface38.

The input device 31 may be a keyboard or a mouse operated by a user ofthe computer, and is used for inputting various kinds of data to thecomputer. The display unit 32 displays various windows and data foroperating the computer. The interface 38 connects the computer to acommunication network, and may be a Network Interface Card (NIC) or amodem.

Programs executing on the computer may be provided from the storagemedium 34 such as a CD-ROM, or may be downloaded through thecommunication network. The storage medium 34 may be set in the drivedevice 33, and programs or data may be loaded to the secondary storagedevice 35 from the storage medium 34 through the drive device 33.

The secondary storage device 35 stores data, programs and othernecessary files.

The memory device 36 loads programs from the secondary storage device 35when starting up the computer. The processor 37 executes variousprocesses in accordance with the programs loaded in the memory device36.

The input device 31, the display unit 32, the drive device 33, thesecondary storage device 35, the memory device 36, the processor 37, andthe interface 38 are connected to a bus B.

FIG. 3 is a block diagram showing functional constituent elements Athrough K of the document processing system of the embodiment of thepresent invention.

As illustrated in FIG. 3, the document processing system includesfunctional constituent elements A through K. A user 110 uses thedocument processing system. An external program 111, which is preparedin advance, is executed according to the hand-written information. Astorage device 112 stores various data. It may be a hard disk drive, aMO drive, or a semiconductor memory drive.

The element A 121 assigns a paper ID to a piece of paper, prints thepaper ID on the paper, and associates the paper ID with a document IDand a process ID. These IDs are described below. The element B 122manages the paper ID, and associates the paper ID with the document IDassigned to a digital document having a specified format. Below, the“digital document” is referred to as “electronic document”.

The element C 123 is a form, or generally, a document on paper (referredto as a paper document), to which a paper ID has been assigned. The user110 writes on the paper to fill out the form or modifies the documentthereon. The element D 124 acquires data and the paper ID from the paperdocument. The element E 125 decodes the paper ID. The element F 126encodes the paper ID. The element G 127 is the paper ID. The element H128 associates the process ID with the external program 111, and managesthe process ID. The element I 129 edits and manages layout informationof the specified format of the electronic document and the process ID.The element J 130 decomposes the hand-written information based onlayout information of the specified format of the electronic documentand the process ID, and stores the decomposed hand-written information.The element K 131 converts the paper ID to the document ID and theprocess ID.

The elements A through K correspond to functions of the components shownin FIG. 1. Specifically, the elements A, B, E are functions given by theform preparation device 108, the element I 129 is the function of thestorage device 100, and the elements D, F, J, K are functions given bythe form processing device 109.

Further, the printing unit, the process-sheet ID association unit, thedocument-sheet ID association unit, and the printing service unit in theclaims correspond to the element A 121. The sheet ID management unitcorresponds to the element B 122. The information acquisition unitcorresponds to the element D 124. The decoding unit corresponds to theelement E 125. The encoding unit corresponds to the element F 126. Theprocess ID management unit corresponds to the element H 128. Thedecomposition storage unit corresponds to the element J. The IDconversion unit corresponds to the element K 131. The coded sheet IDcorresponds to the result of the element F 126 after encoding the paperID.

In the following description, a two-dimensional code, or a code, or acoded paper ID is just abbreviated as “paper ID” where it is apparent inthe context or in figures.

The elements A through K may be realized by either hardware or software.In the following description, components of the elements are expressedas “modules” where appropriate.

As for the several types of IDs mentioned in the above, a paper ID isuniquely assigned to a piece of paper, and different pieces of paperhave different paper IDs. Alternatively, when printing multiple copiesof a page in an electronic document having a certain document ID, it maybe specified that these copies have the same paper ID. The process ID isassigned to a process for processing the hand-written information in thedocument processing system. The document ID is assigned to an electronicdocument, and sometimes referred to as “original document ID” below.

Next, operations of the document processing system of the embodiment ofthe present invention are described in connection with the elements Athrough K.

The external program 111 is created in compliance with the interfacebetween the external program 111 and the document processing system. Theexternal program 111 can be classified into two types.

One type of the external program 111 decomposes the hand-writteninformation into blocks of data according to the format of the paperdocument, and performs fixed-format document processing for the datablocks. The other type of the external program 111 processes thehand-written information itself, and is referred to as “unfixed-formatdocument processing”.

The form processing is one of fixed-format document processing, in whichthe format of the paper document is specified in advance, and fields,into which specified information should be entered, are specified inadvance. Specifically, in an address field, an address should be writtendown, conversely, the item in the address field is treated as anaddress. Therefore, the fixed-format document processing indicates thoseoperations related to specified writing positions.

Thus, the unfixed-format document processing is processing other thanthe fixed-format document processing. In the embodiment of the presentinvention, in the unfixed-format document processing, data files, whichinclude the hand written information, are saved in specified folders.The external program 111 for unfixed-format document processing monitorsthe folders or the database regularly, obtains new files once the newfiles are newly generated, and processes the new files. Alternatively,the external program 111 for unfixed-format document processing mayoperate under an OS that generates an event each time a new file isstored in the folders or the database, and the external program 111catches the event and processes the new file.

For example, the external program 111 for unfixed-format documentprocessing may be a program that superposes hand written information onan electronic document, and sends the superposed electronic document asan attached file to a predetermined person via an email.

In the fixed-format document processing, because the hand writteninformation is stored in a predetermined database in the presentembodiment, the external program 111 for fixed-format documentprocessing monitors the database regularly, obtains new files once thenew files are newly generated, and then processes the new files.Alternatively, the external program 111 for fixed-format documentprocessing may use a database that generates an event each time new dataare registered with the database, and the external program 111 catchesthe event and processes the new file. For example, the external program111 for fixed-format document processing may be a program that processesa questionnaire.

It is not required that both the external program 111 for fixed-formatdocument processing and the external program 111 for unfixed-formatdocument processing be provided, but at least one of them is necessary.

Below, the external program 111 for fixed-format document processing isreferred to as “fixed-format document processing program” and indicatedby 111 a, and the external program 111 for unfixed-format documentprocessing is referred to as “unfixed-format document processingprogram” and indicated by 111 b.

The external program 111 is registered with the document processingsystem in order to be associated with the document processing system.

The components (modules) for registering the unfixed-format documentprocessing program 111 b are included in the element H 128. With thesemodules, processing information is registered.

Here, by “processing information”, it is meant information indicating aninterface between unfixed-format document processing program 111 b andthe document processing system. For example, it may be a name of afolder where data are stored. In this case, the format of the data isspecified beforehand. Because it should be decided which externalprogram 111 is to be selected at the time of printing, as describedbelow, a caption for this operation is also registered. Due to theregistration, a process ID is automatically assigned in the element H128, and is re-used, as described below.

For the fixed-format document processing, a format should be determined.The components (modules) for determining the format are included in theelement I 129. These modules edit and register layout information andprocessing information of each field of the format.

In the present embodiment, the process ID of the fixed-format documentprocessing is registered to be 0 in advance. For example, the layoutinformation may indicate positions of the input fields on the paper. Theprocessing information may indicate which information should beregistered to which data base, whether or not character recognitionshould be performed, and if yes, which kind of characters should beused.

The user 110 who uses the document processing system prints the paperdocument. For this purpose, first, the element A 121 is used to printthe electronic document to be printed. The electronic document is storedas a file, and the title of the file includes an extension indicatingfunctions of the file.

Because the electronic document for the fixed-format document processingincludes a specified extension, from the extension it can be determinedwhich of the fixed-format document processing and the unfixed-formatdocument processing is to be run.

In the unfixed-format document processing, once an instruction forprinting is sent to the aforesaid modules, the modules presentinformation on the external processing program 111 that has beenregistered with the element H 128, and the modules require the user 110to select the processing after hand writing. Depending on the selection,the process ID is determined.

Once the process ID is determined, the modules register the ID of theelectronic document and the process ID with the element B 122 to obtainthe paper ID. After that, the paper ID is printed on the paper document,thereby, the paper document assigned with the paper ID (that is, elementC 123) is obtained.

In the fixed-format document processing, once a printing instruction issent to the modules, different from the unfixed-format documentprocessing, because the process ID is set to be 0, it is not necessaryto require the user 110 to select the processing. The modules registerthe ID of the electronic document and the process ID with the element B122 and obtain the paper ID. After that, the paper ID is printed on thepaper document, thereby, the paper document assigned with the paper ID(that is, element C 123) is obtained.

Due to the above processing, the paper document, the correspondingelectronic document, and the processing to be performed after handwriting are connected with each other. That is, the paper ID is attachedon the paper document, the paper ID is connected to the correspondingdocument ID and process ID. As a result, the paper to be written and theprocessing after writing are registered with the document processingsystem.

The aforesaid original document is the document on which hand-writteninformation has not be written.

Next, a description is given to the operations of the documentprocessing system after hand writing. The user 110 writes on the createdpaper document (that is, the element C 123), and the paper document withthe additional hand writing is read by the element D 124, for example,the scanner 105, or a scanner portion of a multi-function printer.Thereby, the paper ID is obtained by the element E 125 in the element D124. The paper ID and the image are sent to the element K 131.

Alternatively, the user 110 sets the created paper document,corresponding to the element C 123, on a coordinate input device,corresponding to the element D 124, and then writes on the paperdocument. Then, the paper ID is obtained by the element E 125 in theelement D 124. The paper ID and the image are sent to the element K 131.

Because the document ID and the process ID are registered with theelement B 122, and the paper ID has been obtained, in the element K 131,by querying the element B 122 with the obtained paper ID, the documentID and the process ID can be determined.

Next, by querying the element H 128, the storage address of the handwritten information is obtained from the process ID, then the documentID and the image are stored at this storage address.

When the process ID is 0, the fixed-format document processing isperformed. Specifically, the document ID and the image are given to theelement J 130, in the element J 130, by querying the element I 129, thedocument ID, and the layout information and processing information ofthe format can be obtained from the paper ID. Here, the processinginformation may include a storage destination, character recognition ofthe hand written information, or mark recognition.

With the layout information, the image is decomposed into partscorresponding to the input fields. If the scanner 105 is used as theelement D 124, the image of the electronic document is compared with theimage of the hand written paper document so as to obtain the handwritten information only. Alternatively, if the digital PAD 104 is usedas the element D 124, the coordinates of hand writing are obtained.After that, following the processing information, character recognitionor mark recognition is performed, and the results are stored in thedatabase.

The above is a detailed description of the document processing sequence.Next, the elements A through K are described in detail.

First, embodiments of the element A 121 are described.

FIG. 4 is a flowchart showing a first embodiment of the element A 121.In the first embodiment of the element A 121, a stand alone computerprints documents by itself without communication through a communicationnetwork.

In step S101, a layout editor is started up to display an existingelectronic document. The layout editor is capable of reading orselecting the process ID, acquiring the paper ID, displaying, editingand printing the electronic document. This function is realized bycooperative operations of the element B 122, the element F 126, theelement H 128, and the element I 129 (form preparation program) in thecase of a fixed-format document, or a word processor application programin the case of an unfixed-format document.

In step S102, the electronic document is edited when necessary.

In step S103, when the user 110 makes a printing request, the type ofthe electronic document, that is, a fixed-format document or anunfixed-format document, is determined by the extension of the filename.

If the electronic document is an unfixed-format document, the registeredinformation of the external program 111 with the element H 128 ispresented to the user 110 to determine the process ID of the processafter hand writing. If the electronic document is a fixed-formatdocument, once an instruction for printing is delivered, different fromthe unfixed-format document processing, because the process ID isalready set to be 0, it is not necessary to require the user 110 toselect the processing.

In step S104, the paper ID is obtained.

In step S105, the paper ID is encoded into a two-dimensional code.

In step S106, the two-dimensional code is pasted at a specified positionin the electronic document.

In step S107, the electronic document is printed.

When obtaining the paper ID in the above processing, the element B 122is used to associate the paper ID with each page of the electronicdocument, and thus with the process ID; meanwhile, these IDs are managedby a management database. After the paper ID is obtained, the element F126 generates the two-dimensional code from the paper ID.

The generated two-dimensional code is superposed at a specified positionin the electronic document if the position of superposition isspecified, or is superposed at a predetermine position in the electronicdocument if the position of superposition is not specified.

The sequence of operations from acquisition of the paper ID to insertionof the two-dimensional code into the electronic document may be executedautomatically when the user 110 carries out a printing operation byusing the word processor application program. Alternatively, thissequence of operations may be included in a menu, and the user 110 canexplicitly execute the sequence of operations at any time.

If the word processor application program has a macro function, theabove function can be realized by using macros. In this case, theelement B 122 and the element G 127 are software modules. If they areCOM modules of Windows (registered trade mark), they can be called byusing the macro function. If the AddPicture method of theWordObjectLibrary module is utilized, the image can be inserted at anyposition in the electronic document. In this way, the element C 123,that is, the paper document, is printed.

Step S104 corresponds to the step of “associating the sheet ID with adocument ID assigned to a computerized second document”, and the step of“assigning the sheet ID to the first document” in the claims, andoperations of management of the paper ID and document ID.

In step S104, the element B 122 is used to obtain the paper ID,specifically, the element B 122 controls the paper ID, associates thepaper ID with the document ID, and the paper ID with the process ID.

Step S105 corresponds to the step of decoding, and step S107 correspondsto the step of printing in the claims.

FIG. 5 is a flowchart showing a second embodiment of the element A 121.The second embodiment relates to operations of printing an electronicdocument by a printing service after creating and editing the electronicdocument.

In step S201, the user 110 creates an electronic document, in which eachconstituent element (objects) and the characteristics thereof aredescribed.

In step S202, the user 110 starts up a layout editor to display and editan existing electronic document if necessary.

In step S203, using the layout editor, the process ID is selected, andthe paper ID is acquired.

In this step, the element H 128 is used to select the process ID, andthe element B 122 is used to obtain the paper ID. The element H 128 andthe element B 122 associate the paper ID with each page of theelectronic document, and thus with the process ID. Meanwhile, these IDsare managed by a management database.

The selection of the process ID is performed only in the unfixed-formatdocument processing. In the fixed-format document processing, becausethe process ID is already set to be 0, it is not necessary to requirethe user 110 to select the processing.

In step S204, the element F 126 generates a two-dimensional code fromthe paper ID.

In step S205, the two-dimensional code is pasted at a specified positionin the electronic document when the position of pasting is specified, oris pasted at a predetermine position in the electronic document when theposition of pasting is not specified.

In step S206, the electronic document pasted with the two-dimensionalcode, or the storage address of the electronic document is sent to aprinting service by SOAP (Simple Object Access Protocol). When sendingthe storage address of the electronic document, it is preferable thatthe electronic document pasted with the two-dimensional code be storedat an address which the printing service is able to access.

In step S207, it is determined which of the electronic document and thestorage address of the electronic document is to be sent. Differentprocessing is performed depending on the result. If the electronicdocument is to be sent, the electronic document is printed by usingfunctions of the layout editor or a viewer in step S209.

In step S208, if it is determined that the storage address of theelectronic document is to be sent, the electronic document is retrievedfrom the storage address in the printing service.

In step S209, the electronic document is printed by using functions ofthe layout editor or a viewer.

According to the second embodiment, printing is possible even if printerdrivers or other software related to the printing functions are notinstalled in the computer for editing the layout.

The printing service is an environment for printing built on a specifiedcomputer, and this environment is open to other computers. Therefore,any computers able to access to the printing service are allowed toprint electronic documents using the printing function. In other words,even a computer without the printing environment can deliver aninstruction of printing an electronic document.

In addition, because editing is not necessary in the printing service,it is sufficient to install a viewer having printing commands in acomputer on which the printing service is running.

FIG. 6 is a flowchart showing a third embodiment of the element A 121.The third embodiment relates to operations of printing an electronicdocument by using a printing service without editing the electronicdocument.

In step S301, the user 110 creates an electronic document.

In step S302, the electronic document, or the storage address of theelectronic document is sent to a printing service by SOAP (Simple ObjectAccess Protocol) together with the process ID.

When sending the storage address of the electronic document, it ispreferable that the electronic document to be printed be stored at anaddress which the printing service is able to access.

If the electronic document is an unfixed-format document, the element H128 obtains a list of the process IDs, and requires the user 110 toselect the process ID to be sent. If the electronic document is afixed-format document, the process ID is 0.

In step S303, it is determined which of the electronic document and thestorage address of the electronic document is to be transmitted.Different processing is performed depending on the result.

If it is determined that the electronic document is to be transmitted,the routine proceeds to step S305.

If it is determined that the storage address of the electronic documentis to be transmitted, the routine proceeds to step S304.

In step S304, the electronic document is retrieved from the storageaddress in the printing service.

In step S305, if the electronic document is transmitted, or after theelectronic document is retrieved, the computer on which the printingservice is running acquires the paper ID.

In this step, the element B 122 is used to obtain the paper ID,associate the paper ID with each page of the electronic document, andthus with the process ID. Meanwhile, these IDs are managed by amanagement database.

In step S306, the element F 126 generates a two-dimensional code fromthe paper ID.

In step S307, the two-dimensional code is pasted at a specified positionin the electronic document when the position for pasting is specified,or is pasted at a predetermine position in the electronic document whenthe position of pasting is not specified.

In step S308, an image file for printing the electronic document pastedwith the two-dimensional code is created and is printed on paper.

The steps of acquiring the paper ID, creating the two-dimensional code,pasting the two-dimensional code, and printing the electronic documentare executed by using functions of a layout editor installed in thecomputer on which the printing service is running.

According to the third embodiment, printing an electronic document ispossible even if a layout editor is not installed in the computer onwhich the electronic document is created. The printing service isrunning on a specified computer, and if the layout editor is installedin this specified computer, other computers can use this function toprint electronic documents. In other words, even computers without thelayout editor are able to deliver instructions of printing electronicdocuments.

FIG. 7 is a flowchart showing a fourth embodiment of the element A 121.The fourth embodiment relates to operations of printing an electronicdocument by using a printing service without editing the electronicdocument.

In step S401, the user 110 creates an electronic document.

In step S402, the electronic document, or the storage address of theelectronic document is sent to a printing service by SOAP (Simple ObjectAccess Protocol) together with the process ID.

When sending the storage address of the electronic document, it ispreferable that the electronic document to be printed be stored at anaddress which the printing service is able to access.

If the electronic document is an unfixed-format document, the element H128 obtains a list of the process IDs, and requires the user 110 toselect the process ID to be sent. If the electronic document is afixed-format document, the process ID is 0.

In step S403, it is determined which of the electronic document and thestorage address of the electronic document is to be transmitted.Different processing is performed depending on the result.

If it is determined that the electronic document is to be transmitted,the routine proceeds to step S405.

If it is determined that the storage address of the electronic documentis to be transmitted, the routine proceeds to step S404.

In step S404, the electronic document is retrieved from the storageaddress in the printing service.

In step S405, if the electronic document is to be transmitted, or afterthe electronic document is retrieved, the electronic document isconverted into an image file for printing use.

In step S406, the computer, on which the printing service is running,acquires the paper ID.

In this step, the element B 122 is used to obtain the paper ID,associate the paper ID with each page of the electronic document, andthus with the process ID. Meanwhile, these IDs are managed by amanagement database.

In step S407, the element F 126 generates a two-dimensional code fromthe paper ID.

In step S408, the two-dimensional code is also converted into an imagefile for printing use. The image file of the two-dimensional code ispasted at a specified position in the image file of the electronicdocument when the position for pasting is specified, or is pasted at apredetermine position in the image file of the electronic document whenthe position of pasting is not specified.

In step S409, the image file of the electronic document pasted with theimage file of the two-dimensional code is printed on paper.

According to the fourth embodiment, printing an electronic document ispossible even if a layout editor is not installed in the computer onwhich the electronic document is created. The printing service isrunning on a specified computer, and if the layout editor is installedin this specified computer, other computers can use this function toprint electronic documents. In other words, even computers without thelayout editor are able to deliver instructions of printing electronicdocuments.

In addition, because the electronic document itself is not edited in theprinting service, but an image pasting operation is performed by usingimage files for printing use, it is sufficient to install a viewer thathas functions of outputting files by using printing commands in thecomputer on which the printing service is running.

Further, before pasting the two-dimensional code to the electronicdocument, the electronic document and the two-dimensional code areconverted into respective image files for printing use first, and theimage file of the two-dimensional code is pasted to the image file ofthe electronic document in the printing service, therefore, no matterwhat kind of file the original electronic document is, pasting of thetwo-dimensional code and outputting of the electronic document pastedwith the two-dimensional code can be performed in the same way. This isan advantage of the fourth embodiment.

FIG. 8 is a flowchart showing a fifth embodiment of the element A 121.The fifth embodiment relates to operations of printing an electronicdocument by using a printing service.

In step S501, the user 110 uses a layout editor to create and edit anelectronic document.

In step S502, a printing image or a page descriptive language sequenceis created from the electronic document. The printing image and the pagedescriptive language sequence are not dependent on the relevantplatform. Here, a printing image is an image of a paper document that isactually printed on paper.

Below, for simplicity, the word “printing image” is used to indicateboth the printing image and the page descriptive language sequence.

When an existing electronic document is opened directly without anyediting, the printing image may be created from the electronic documentby using the printing commands of a layout viewer.

In step 503, the printing image of the electronic document created inthis way is sent to a printing service by using SOAP (Simple ObjectAccess Protocol), together with the process ID selected by the user 110,and the document ID indicating the storage address of the electronicdocument.

In step S504, the computer, on which the printing service is running,acquires the paper ID after receiving the printing image, the processID, and the document ID.

In this step, the element B 122 is used to obtain the paper ID,associate the paper ID with each page of the electronic document, andthus with the process ID. Meanwhile, these IDs are managed by amanagement database.

In step S505, the element F 126 generates a two-dimensional code fromthe paper ID.

In step S506, the two-dimensional code is transformed according to theform of the printing image of the electronic document received before,and is pasted at a specified position in the electronic document whenthe position of pasting is specified, or is pasted at a predetermineposition in the electronic document when the position of pasting is notspecified.

In step S507, the printing image of the electronic document pasted withthe two-dimensional code is printed on paper.

According to the fifth embodiment, it is possible to print theelectronic document pasted with the two-dimensional code even when theoriginal electronic document is stored at an address which the printingservice is unable to access. In addition, although the printing servicepastes the two-dimensional code to the electronic document afterreceiving the printing image of the electronic document, because alayout editor is not used for pasting the two-dimensional code, it isnot necessary to install the layout editor in the computer.

When printing an exiting electronic document, which does not be edited,it is sufficient if the computer, on which the electronic document wascreated, is able to deliver a printing instruction for creating aprinting image, and it is not necessary to install a layout editorhaving editing functions in the computer.

In other words, it is possible to paste a two-dimensional code to anelectronic document even when a layout editor is not installed in eitherof the two computers in communication. For example, in the case of a PDFfile, it is not necessary to install an editing application like Acrobat(registered trade mark) of the Adobe company in the computer on whichthe electronic document is created, and it is sufficient to install anapplication like Acrobat Reader (registered trade mark) that is able todeliver a printing command.

Next, embodiments of the element B 122 are described.

When other elements shown in FIG. 3 request registration of a paper IDin order for outputting onto paper, the element B 122 receives at leastdocument information that uniquely specifies an electronic document,assigns a unique paper ID arranged in the management table correspondingto the document information, and stores the document information to themanagement table.

FIG. 9 is a view showing an example of a management table used in theelement B 122 for associating a paper ID with a document ID.

FIG. 10 is a view showing another example of the management table usedin the element B 122 for associating a paper ID with a document ID.

In FIG. 9 and FIG. 10, each line, for example, the line 145, is a unitof the document information corresponding to the registered paper ID,and each column includes registered elements of the paper IDs.

Listed in column 141 are the assigned paper IDs. In column 142, there isthe document information for uniquely specifying an electronic document.For example, in FIG. 9, path names are stored in the column 142 as thedocument information. If electronic documents in different computers aredealt with, as shown in FIG. 10, network path names (URI: UniformResource Identifier) are stored in the column 142 as the documentinformation. When dealing with electronic documents each having multiplepages, the document information is provided in each page, and the column142 also includes information of page numbers.

In addition, as shown in FIG. 10, if information of page numbers is notstored in the column 142, this information can be stored in the column146. In this case, an electronic document is uniquely specified bycolumns 142 and 146.

A storage address of an electronic document may be represented by amanagement ID if the electronic document is stored in the same documentmanagement system. As shown in FIG. 10, the electronic documents may bestored in a different system or in multiple systems as long as thestored electronic documents can be identified. In addition, whenaccepting registration requests, a paper ID may be assigned after theelectronic document itself is received, given a file namedistinguishable from others, and saved.

In the management table, a column 143 stores process IDs indicatingprograms to be used to process a paper document and hand writteninformation thereon.

In addition, column 144 stores information of applications used tocreate electronic documents. There may be two or more columns like thecolumn 144 for storing attribute information related to electronicdocuments and paper documents.

When a paper ID is specified and the management information is requested(referred to as “reference request”), the element B 122 searches for anappropriate paper ID from the paper IDs in the column 141 in FIG. 9. Theelement B 122 transmits, to the requester, an item of the documentinformation in the column 142 corresponding to the obtained paper ID,which uniquely specifies an electronic document. At this time, theelement B 122 may also transmits attribute information in the column 143and the subsequent columns at the same time, but at least theinformation in the column 142 should be transmitted, and in FIG. 10, atleast the information in the columns 142 and 146 should be transmitted.

When the element B 122 is a program, the contents of the managementtables in FIG. 9 and FIG. 10 may be stored in the memory used by theprogram in execution. Alternatively, the contents of the managementtables may be stored as a file in a file system at certain timing.Further, the contents of the management tables may be stored in adatabase as digital information. In this case, one paper ID and thecorresponding attribute information form one record, and the paper IDmay be used as a key for storing or retrieving document-relatedinformation, thereby enabling association of the paper ID with theelectronic document and enabling management of the information.

In this way, the management table controlled by the element B 122 isstored not only in the memory used by executing programs, but also in anon-volatile storage device such as a hard disk as a file. Therefore,even when the power of the element B 122 is turned off, the associationinformation may be evoked again for use.

In addition, information of the correspondence relation between thepaper IDs and the electronic documents held in the management table isstored in the non-volatile storage device as a database able to beaccessed via a communication network. Thereby, it is possible to providemore than one elements B 122 in the system, which refers to thecorrespondence relation information in the database.

The non-volatile storage device described here corresponds to thenon-volatile storage unit in the claims.

Furthermore, the element B 122 may be provided in a device independentfrom the device provided with the other elements, and the device havingthe element B 122 may be connected with the device having the otherelements by a communication network so that the device having the otherelements can access the device having the element B 122. With thisconfiguration, the element B 122 functions as RPC (Remote ProcedureCall) or a Web service, and accepts requests for registration or forreference from other elements via the communication network.

In this way, the element B 122 is connected with other elements via thecommunication network, and if the element B 122 is the only element thatissues identifiers to paper documents in the system, the element B 122can accept requests for registration or for reference from otherelements distributed on the communication network.

Further, when information of the electronic document is received, evenif the same file name of different electronic documents exists on thecommunication network, the electronic documents can be distinguished ifa path name, or a name of the computer storing the electronic documents,is included in the file name.

After the electronic document to be registered is received and stored,the name of the stored electronic document is associated with theassigned paper ID in management, therefore, it is possible todistinguish the same electronic document name of different registrationsin management.

Next, a description is given to embodiments of the element C 123, whichcorresponds to a paper document assigned with a paper ID.

FIGS. 11A and 11B are views showing an example of a paper document.

The paper document shown in FIG. 11A or FIG. 11B is a registration form40 for entrance to or resignation from a club. This paper document isobtained by printing a corresponding electronic document, which iscreated and managed in a computer, for example, onto standard paper.

As illustrated in FIG. 11A, contents 46, a two-dimensional code 45, andfour timing marks 41, 42, 43, 44 are printed on the registration form40.

The contents 46 are defined by the corresponding electronic document,the two-dimensional code 45 is mechanically readable, and the timingmarks 41, 42, 43, 44 are used for position alignment when reading handwritten information. The number of the timing marks is not limited tofour, and the shapes, sizes, and colors of them are not limited, either.Moreover, depending on the situation, the timing marks may also beomitted.

As shown in FIG. 11B, one uses a writing tool to fill out theregistration form 40, and the filled-out registration form is indicatedto be a registration form 47.

FIG. 12 is a view showing another example of the paper document. In FIG.12, data are embedded into contents of a paper document 59.

As illustrated in FIG. 12, contents (information) and four timing marks48, 49, 50, 51 are printed on the paper document 59. A portion 52 of thecontents is enlarged (the enlarged portion is indicated by a referencenumber 53). In the enlarged portion 53 of the contents, there are anidentifier 54, a code frame 55, which also acts as alignment dots, anddata dots 56.

In this example, it is described that the element C 123 corresponds to adocument printed on paper. It should be noted that the element C 123 isnot limited to this, but may be any thing on which one can write, forexample, a sheet of plastic film and others.

Next, a description is given to embodiments of the element D 124, whichobtains the hand written information from the paper document. Forexample, the element D 124 may be an image capturing device using anarea CCD as an image pick-up element, specifically, a digital camera.

When an image of a piece of A4 size paper is captured, at a time, by anarea CCD having about 300 million pixels (2048×1536 pixels), theequivalent effective resolution is about 170 dpi (dots per inch). Whencapturing an image including a great deal of information, such as thetwo-dimensional code, and decoding the two-dimensional code, a highresolution is necessary, and sometimes the resolution of 170 dpi is notsufficient. It is possible in principle to attempt to increase thedegree of integration of the area CCD so as to increase the resolution,but this is limited by the technology of semiconductor processing.

To solve this problem, in this embodiment, the image capturing deviceuses the area CCD as the image pick-up element, and is equipped with azoom lens whose magnification is changeable. Further, the imagecapturing device is moved by a driving unit, and the image capturingdevice can zoom in on regions of an object (for example, a paperdocument on a piece of paper) to capture images of these regions. Then,the images of these regions are combined into one image having a highresolution. In this way, it is possible to read the two-dimensionalcode.

FIG. 13 is a perspective view of a configuration of the image capturingdevice according to the embodiment of the present invention.

The image capturing device in FIG. 13 includes an image pick-up unit201, a driving unit 202, a supporting column 203, an interface 204, acontrol unit 205, a stand 206, and two operational switches 207.

The driving unit 202 drives the image pick-up unit 201 to move asindicated by the dashed line. The supporting column 203 supports theimage pick-up unit 201 and the driving unit 202. The control unit 205controls operations of the image capturing device, and also supports thesupporting column 203 and the stand 206. The control unit 205corresponds to the “control unit”, the “combination unit”, and the“detection unit” in the claims. The operational switches 207 are usedfor operating the image capturing device. The regions A through Iindicated by dashed lines on the paper document 208 are the partialregions, and the image capturing device picks up an image of one of thepartial regions each time.

FIG. 14 is a block diagram showing configurations of the image pick-upunit 201, the driving unit 202, and the control unit 205.

The control unit 205 is a small size board computer unit. It includes asystem controller 211, a CPU (Central Processing Unit) 210, a SDRAM(Static Dynamic Random Access Memory) 212, a PCI (Peripheral ComponentsInterconnects) bus 213, a PCMCIA (Personal Computer Memory CardInternational Association) 214, an I/O device 215, an IDE (IntegratedDrive Electronics) 217, a PCI/ISA bridge IDE USB transformationinterface 218, a USB (Universal Serial Bus) 219, a hard disk 216, an ISA(Industry Standard Architecture) bus 220, an I/O controller 221, aserial bus 222, a parallel bus 234, and a LAN (Local Area Network)interface 235.

The control unit 205 controls operations of the image pick-up unit 201,processes, edits, or records the images transmitted from the imagepick-up unit 201, and communicates with external devices which areconnected through the interface 204.

The interface 204 may be a versatile interface for personal computers,for example, RS-232C, USB, IEEE 1394, IrDA, or a network card.

In this embodiment, it is described that the image capturing device iscontrolled by the control unit 205, but the operations performed by thecontrol unit 205 may also be performed by a MPU (Micro-Processor Unit)236 in the image pick-up unit 201.

The driving unit 202 includes stepping motors 255, 256, and rotaryencoders 244, 257 for detecting a rotation angle, a detection circuit258, and a driving circuit 243.

If the rotation positions of the stepping motors 255, 256 are calibratedrelative to a reference position, the rotary encoders 244, 257 can beomitted.

FIG. 15 is a diagram showing a configuration of the image pick-up unit201.

As illustrated in FIG. 15, the image pick-up unit 201 includes a CDS(Correlated Double Sampling) 241, an AD converter 242, an IPP (ImagePre-Processor) 281, a memory 240, a TG (Timing Generator) 280, a MPU236, an I/O controller 237, an I/O device 261, fixed lenses 267, 268, azoom lens 266, an aperture diaphragm 265, a shutter 264, a focus lens263, and an image pick-up element 262.

Here, the I/O device 261 represents the serial bus 238 and the USB 239in FIG. 14.

Light from an object (for example, a paper document on a piece of paper)passes through the fixed lenses 267, 268, the zoom lens 266, theaperture diaphragm 265, and the shutter 264 controls the period ofexposure, and finally, an image of the object is formed on the imagepick-up element 262.

Image signals from the image pick-up element 262 are sampled in the CDS241, converted into digital signals in the AD converter 242. The TG 280generates signals having specified timing, and the CDS 241 and the ADconverter 242 operate according to the timings generated by the TG 280.

Afterward, the image signals are processed in the IPP 281, for example,for aperture correction, or image compression, and then are stored inthe memory 240.

Operations of the above components of the image pick-up unit 201 arecontrolled by the MPU 236. The image pick-up unit 201 is connected withexternal devices through the I/O controller 237 and the I/O device 261so as to input or output image data or transmit image pick-up controlsignals.

Next, with reference to FIG. 14 again, a description is made of thesequence of dividing an object into partial regions and capturing imagesof these partial regions.

First, the control unit 205 directs the I/O device 215 to apply a pulsedvoltage to the driving circuit 243 of the driving unit 202, and feedsback the rotation angle detected by the detection circuit 258 until therotation angle becomes a predetermined value.

Second, the control unit 205 directs the USB 219 to send image pick-upcontrol signals to the USB 239 of the image pick-up unit 201.

Third, the above two steps are repeated if necessary. If image transferis instructed, the images from the image pick-up unit 201 aretransferred to the control unit 205 via the USB 239. The control unit205 transmits the image to the communication network via the LANinterface 235.

Fourth, the paper document 208 (as shown in FIG. 13) is illustrated inFIG. 16. As shown in FIG. 16, identification information, such as atwo-dimensional code 290, is printed on the paper document 208.

The two-dimensional code 290, the so-called coded paper ID, is decodedby the element E 125. If the decoding is successful, the paper ID can beobtained. Then, data type, paper ID, and information on images aretransmitted to the element K 131 as data in the XML form (eXtensibleMarkup Language).

FIG. 17 is a flowchart showing operations of the image capturing devicefrom capturing the image of the paper document to associating thecaptured image with the result of decoding the two-dimensional code.

In step S1401, the image capturing device adjusts the zoom lens so thatthe object (the paper document) and the imaging plane of the imagepick-up unit 201 of the image capturing device are nearly parallel toeach other, and in this state, the image capturing device shoots thewhole paper document.

In step S1402, from the captured image of the whole paper document, theimage capturing device detects a code region, that is, the regionprinted with the two-dimensional code.

In step S1403, the image capturing device calculates the coordinates andthe center of the code region. In this step, if the code region isdefined beforehand, this step can be completed at a high speed.Alternatively, instead the code region, the center of a circumscribedrectangle of the code region may be used.

In step S1404, using the coordinates and the center of the code regionobtained above, the image capturing device calculates the magnificationand movement caused by the driving unit 202 by linear operations so asto enlarge the image of the data identification information. Here, themovement caused by the driving unit 202 may be calculated in a pandirection and in a tilt direction with the optical axis to be the rollaxis.

In step S1405, based on the calculation results, the driving unit 202drives the image pick-up unit 201 to move.

In step S1406, the image pick-up unit 201 starts to pick up an image atthe newly set position.

In step S1407, the element E 125 decodes the two-dimensional code in thecaptured image.

In step S1408, the image capturing device relates the image of the wholepaper document to the decoded result.

Therefore, an image of the code region is obtained at a high resolution,and the paper ID of high confidence can be assigned to the image of thewhole paper document.

Next, the image capturing process is described in detail.

As mentioned above, in the image capturing device, the driving unit 202drives the image pick-up unit 201 to capturing images of partial regionsof the paper document (it is referred to as “divisional image pick-up”below), and then the images of the partial regions of the paper documentare combined into one image.

In the divisional image pick-up, as illustrated in FIG. 13, the drivingunit 202 drives the image pick-up unit 201 to move as indicated by thedashed line in FIG. 13 to capture images of the partial regions A, B,and C.

FIG. 18 is a schematic view showing movement of the image pick-up unit201.

As illustrated in FIG. 18, the image pick-up unit 201 is driven to swingup and down (rotate) to cover all of the regions A through I.

In FIG. 18, the image pick-up unit 201 is moved to shoot the paperdocument 208, for example, regions 301 and 302 of the paper document208.

As mentioned above, in order to capture a high resolution image of theobject, the scope (or angular region) covered by the image pick-up unit201 each time should be made as small as possible so as to shoot at morepositions. After shooting, all of the regions of the object are includedin the obtained divisional images.

FIG. 19 is a view for schematically showing the divisional images.

In FIG. 19, there are six divisional images of alphabet, and it is clearthat the letters at lower positions become smaller.

FIG. 20 is a view for schematically showing combination of thedivisional images.

The image shown in FIG. 20 is obtained by combining the six divisionalimages in FIG. 19.

The distortion of the final image shown in FIG. 20 occurs not only whencombining divisional images by the way described above, but also occurswhen shooting the whole paper document at a time without image division.

FIG. 21 is a view showing a corrected image.

The image in FIG. 21 is obtained by correcting the distortion andeliminating noise in the image shown in FIG. 20.

To perform such correction, a reference image is defined, which isobtained by shooting the object with the image pick-up unit 201 beingright in front of the object, and a projecting transformation matrix fortransformation from the reference image to other images is found. Withthis matrix, the other images are re-arranged.

For simplicity, the correction is described with reference to FIG. 18for picking up images of the two regions 301 and 302.

FIG. 64 shows equations (1) through (4) used in distortion correction.

A point in the region 301 and a point in the region 302 can berepresented by the following equation (1).

If the surface of the object is a plane, the point in the region 301 andthe point in the region 302 satisfy the following equation (2) andequation (3).

In equation (2) and equation (3), the quantity h can be given by amatrix in equation (4).

The matrix in equation (4) is a projecting transformation matrix, andwhen two shooting positions are the same, this relation does not change.Thus, quantities h1 through h8 may be calculated from given groups (u1,v1), (u2, v2).

With equations (1), (2), and (3), points in the region 302 are relatedto points in the region 301. With the region 301 as a reference andusing the obtained correspondence relation, the pixels of the image ofthe region 302 can be mapped to the image of the region 301. If thereare three or more regions on the paper document, they are mapped in asimilar way.

The projecting transformation matrices are found first, and with theprojecting transformation matrices, the images of regions other than thereference region are transformed to front-side images, that is, an imageof a region right in front of the image pick-up unit 201. In this way,the front-side images are obtained, and by combining these transformedfront-side images, an image without distortion as shown in FIG. 21 canbe obtained.

In the above, the configuration and operation of the image capturingdevice according to the embodiment of the present invention aredescribed. Below, specific examples of the image capturing device aredescribed.

FIG. 22 is a perspective view showing an example of the image capturingdevice.

As illustrated in FIG. 22, in the image capturing device, an imagepick-up unit 310 is provided for reading a code, such as thetwo-dimensional code, in addition to the image pick-up unit 201 forreading the paper document in the configuration shown in FIG. 13.

Because the image pick-up unit 310 is just for capturing an image of thesmall code region, the image pick-up unit 310 may be a compact imageinput device much cheaper than the image pick-up unit 201.

In addition, decoding results of the code 311 in the image captured bythe image pick-up unit 310 may be stored in the image capturing devicein correspondence to the image captured by the image pick-up unit 201.

FIG. 23 is a perspective view showing another example of the imagecapturing device.

As illustrated in FIG. 23, in the image capturing device, instead of theimage pick-up unit 310 in FIG. 22, a paper ID reader 312 is provided forreading the two-dimensional code. The paper ID reader, 312 may be animage pick-up unit used in a bar code reader.

FIG. 24 is a perspective view showing another example of the imagecapturing device.

In the image capturing device illustrated in FIG. 24, the code 311 isprovided on an underlying sheet 313 of the paper document 208. Thesupporting post 203 is on a stand 315.

With this configuration, code can be assigned to paper without a code.By using a printer, paper can be printed again by assigning the code.

FIG. 25 is a perspective view showing another example of the imagecapturing device.

In the image capturing device illustrated in FIG. 25, a code 314 isprovided on a plate member of the stand 315. The plate member can be putin the stand 315, and the plate member corresponds to the coded paper IDdisplay unit as defined in the claims.

FIG. 26 is a perspective view showing another example of the imagecapturing device.

In the image capturing device illustrated in FIG. 26, a projecting unit316 is attached to the image pick-up unit 201, and a code 317 can beprojected in the indicated projecting region. Users can project the code317 as desired. The code 317 may be editable digital data.

The projecting unit 316 may also be installed at other positions insteadthat shown in FIG. 26. The projecting unit 316 may be an opticalprojecting device, for example, the projecting unit 316 may be a typicalliquid crystal projector including a light source, a condensing lens,mirrors, a liquid panel, and a projecting lens, or a DLP (Digital LightProcessing) projector using DMD (Digital Mirror Device). Preferably, theprojecting unit 316 is a compact one.

FIG. 27 is a perspective view showing another example of the imagecapturing device.

In the image capturing device illustrated in FIG. 27, a portableterminal 318 having a display, such as a PDA (Personal DigitalAssistant) having a liquid crystal display, is provided in the shootingregion as indicated by the dashed lines. The code can be displayed bythe portable terminal 318, and thereby, the code associated with thepaper document to be processed can be assigned to the paper document.The portable terminal 318 corresponds to the display unit in the claims.

It should be noted that a driving unit is not indispensable in theconfigurations shown in FIG. 24 through FIG. 27.

In the above, the element D 124 is embodied to be an image capturingdevice. Below, a description is made of the element D 124 embodied to bea written information input device.

FIG. 28 is a block diagram of a written information input device 60.

The written information input device 60 includes a coordinate input unit74 for acquiring written information on paper, a paper ID reader 75 foridentifying digital information associated with information printed onpaper, a storage unit 71 for storing the obtained information, acommunication unit 72 for transmitting the obtained and storedinformation to a computer, and a control unit 73 for controlling thecoordinate input unit 74, the paper ID reader 75, the storage unit 71,and the communication unit 72.

The coordinate input unit 74 may be a digitizer, which successivelyobtains coordinates of hand writing. It is known that some digitizerswork by electromagnetic induction, and some others work in the same wayas an ultrasonic distance meter. In either of them, a sensor and adedicated pen are used to detect writing movement on paper to measurethe written information and writing coordinates, which are coordinatesof the track of writing on paper. A collection of the coordinatesmeasured in this way constitutes the written information.

The end of the dedicated pen may be a true pen able to write on paper,and it is preferable that a real writing track appears on paper whilethe writing coordinates are measured electrically.

The paper ID reader 75 is able to read and decode a coded paper ID, forexample, printed on paper by a printer. The paper ID reader 75 may be atwo-dimensional code reader, or a one-dimensional bar-code reader, whichare well known. If the two-dimensional code reader is used as the paperID reader 75, and the well known QR code (registered trademark) is usedfor coding, the paper ID reader 75 can identify at most (decimal) 4296pieces of information. This capacity is sufficiently large foridentifying documents printed by users in a usual way.

The storage unit 71 may be a hard disk, or a nonvolatile memory, or adetachable memory such as a Compact Flash disk (registered trademark)for storing the writing coordinates, or printing conditions information,or electronic document information obtained by the paper ID reader 75.

The communication unit 72 transmits the stored information to a computerby communication networks such as Ethernet (registered trademark) orradio LAN, or by USB, bluetooth, or serial connection technologies. Thecommunication unit 72 may also be omitted, for example, if the storageunit 71 is directly connected to the computer and able to directlycommunicate with the computer.

The storage unit 71, the communication unit 72, the control unit 73, thecoordinate input unit 74, and the paper ID reader 75 and others may beintegrated together, or be separate from each other. When they areintegrated together, by using a battery to power the written informationinput device 60, the device 60 is portable and can be carried to anyplace. It is preferable to make the written information input device 60monolithic and portable, and in this way, writing on printed paper isnot restricted on desks in one's office.

FIG. 29 is a perspective view showing the appearance of the writteninformation input device 60.

In FIG. 29, the paper ID reader 75 is implemented to be atwo-dimensional code reader 62. Text 65 is printed on a piece of paperset on the written information input device 60. The paper ID 63 is alsoprinted on the paper in advance.

A coordinate acquiring unit 61 is implemented to be an ultrasonicdigitizer. By using a dedicated pen 66, the actual writing tracks appearon the paper while coordinates of the writing tracks are electricallymeasured.

FIG. 30 is a flowchart showing operations of a portable digitizer as animplement of the written information input device 60.

In step S601, when writing starts with the dedicated pen, the writteninformation input device 60 detects contact of the end of the pen on thepaper (this is called pen-down).

In step S602, the dedicated pen reads the writing coordinates bydetecting the pen-down.

In step S603, the two-dimensional code reader in the written informationinput device 60 reads the two-dimensional code while the dedicated penis reading the writing coordinates or while the dedicated pen is beingin contact with the paper.

In step S604, the written information input device 60 decodes thetwo-dimensional code to obtain the paper ID.

In step S605, such obtained writing coordinates and the paper ID arethen stored in the storage unit.

In step S606, the communication unit transmits the stored information toa computer when necessary.

FIG. 31 is a table showing data stored and transmitted by the writteninformation input device 60.

Items shown in FIG. 31 are “device identifier”, “paper ID”, “writing Xcoordinate”, “writing Y coordinate”, and “Pen Up/Down”.

The label “device identifier” represents the number for identifyingdifferent written information input devices 60. The label “paper ID”represents the paper ID read by the written information input device 60.The labels “writing X coordinate” and “writing Y coordinate” representthe X and Y coordinates of the written information. The label “PenUp/Down” represents up-movement or down-movement of the dedicated pen.

The writing coordinates and the paper ID are transmitted in such a form.

Below, a description is made of the element D 124 embodied to be animage input device. In the image input device, a scanner is used tocapture an electric image of the paper document after the writing downon the paper, and the data are transmitted to the element E 125.

FIG. 32 is a block diagram showing an image input device 88.

The image input device 88 includes a control unit 82 for controllingcomponents of the device 88, a storage unit 83 for storing programs orimage data, a scanner controller 81 for communicating with andcontrolling an external scanner 80 or a MFP 86, an image processing unit84 for processing image data, and a network controller 85 forcommunicating with and controlling an external communication network.

A scanner 80 is directly connected to the image input device 88, or aMFP 86 is connected to the image input device 88 via a communicationnetwork 87. From the scanner 80 or the MFP 86, images can be input tothe image input device 88.

FIG. 33 is flowchart showing the operation of the image input device 88.In the operation shown in FIG. 33, the image input device 88 acquirescoordinates and transmits data to the element K 131.

In step S701, the image input device 88 reads an image. Specifically,the scanner 80 or the MFP 86 reads an image of the paper document aftera user finished writing on the paper. For example, the conditions forimage reading are a 600 dpi, binary (black and white) image. Usually, ascanner has a function to convert an image into a binary image.

In step S702, the obtained image of the paper document is stored as animage file in a specified folder of the storage unit 83, and is put in aqueue for the next processing.

The scanner controller 81 controls storing the obtained image of thepaper document as an image file in a specified folder of the storageunit 83. When a button on the scanner 80 or the MFP 86 is pressed, thescanner controller 81 detects that the image is to be transmitted fromthe scanner 80 or the MFP 86, and stores the transmitted data in aspecified folder of the storage unit 83.

In step S703, the image processing unit 84 monitors the folder of thestorage unit 83, in which the data transmitted from the scanner 80 orthe MFP 86 are stored, and image erecting processing is performedsequentially from the first image in the queue.

In step S704, the image processing unit 84 extracts a region includingblack pixels surrounded by white pixels (region extraction processing).

In step S705, the image processing unit 84 decodes image data in theextracted region.

In step S706, the image processing unit 84 determines whether thedecoding is successful.

If the decoding is successful, the image processing unit 84 obtains thepaper ID, and then finishes the routine.

If the decoding is not successful, the image processing unit 84 finishesthe routine.

The flowchart in FIG. 33 illustrates operations corresponding to thestep of acquiring information as defined in the claims. The paper IDobtained in this way is converted into a document ID and a process ID byusing a management table as described below. This conversion processcorresponds to the step of “converting the sheet ID to the document IDand the process ID” as defined in the claims.

In addition, in step S702, the image data are put in a queue for nextprocessing. The reason of using a queue is that when a large number ofimage files are input by using ADF (Auto Document Feeder) and theseimage files are processed simultaneously, the work load of the imageprocessing unit 84 increases, thereby the operations of the imageprocessing unit 84 may become unstable, and the operation speed maydecrease.

The image erecting processing in step S703 is for erecting the imagesread by the scanner, whose orientation is unknown. This processing isneeded because in the next step S704, the processing of region selectionis for selecting a region in an erected image.

FIG. 34 is a flowchart showing the image erecting processing.

In step S801, OCR is performed for the images placed in the queue inorder, and the OCR results are stored.

In the loop including step S802, S803, and S801, OCR is performed torotate the image by 90 degrees each time, and the results are alsostored.

In step S804, after the OCR treatment of rotating the image by 90degrees each time, the image orientation is determined by using the OCRconfidence, which is a measure of OCR results. The OCR confidencecorresponds to the likelihood of the recognized characters, and the OCRconfidence used in determining the image orientation is an average ofthe character OCR confidence. A direction, in which the average OCRconfidence is maximum, is determined to be the erecting direction theimage.

After this image erecting processing, in step S704 in FIG. 33, regionextraction is performed for detecting and decoding the two-dimensionalcode including the paper ID. In the region extraction processing, blackpixels enclosed by white pixels in the image are detected.

FIG. 35 is a flowchart showing the region extraction processing.

In step S901, adjacent black pixels are labeled by scanning the pixelsfrom an origin located at the left-upper corner of the image. Whenlabeling a target pixel, if the pixels at the left-upper, upper, andleft positions relative to the target pixel are all white pixels, andthe target pixel is a black pixel, a new label is assigned to the targetpixel.

If the surrounding pixels are not all white pixels, the target pixel isassigned the label of the black pixel at either the left-upper, or theupper, or the left position relative to the target pixel.

FIG. 36 is a diagram for schematically showing the operation of labelingthe black pixels.

In FIG. 36, it is assumed that each of the squares having a figuretherein represents a black pixel. Below, pixels 90, 91, 92, and 93 areused to describe the pixel labeling process. Here, for example, pixels90, 91, 92, and 93 are scanned sequentially.

First, considering the pixel 90, because the pixels at the left-upper,upper, and left positions relative to the pixel 90 are all white pixels,and the target pixel 90 is a black pixel, a label “1” is given to thepixel 90.

Next, considering the pixel 91, the pixels at the left-upper and upperpositions relative to the pixel 91 are white pixels, but the pixel 90 atthe left position relative to the pixel 91 is not a white pixel,therefore, the target pixel 91 is given the label of a black pixel ateither the left-upper, or upper, or left position relative to the targetpixel 91. For example, the label “1” of the pixel 90 is given to thepixel 91.

Next, considering the pixel 92, because the pixels at the left-uper,upper, and left positions relative to the pixel 92 are all white pixels,and the target pixel 92 is a black pixel, a new label “2” is given tothe pixel 92.

Next, considering the pixel 93, the pixels at the left-upper and upperpositions relative to the pixel 93 are white pixels, but the pixel 92 atthe left position relative to the pixel 93 is not a white pixel,therefore the target pixel 91 is given the label of a black pixel ateither the left-upper, or upper, or left positions relative to thetarget pixel 93. For example, the label “2” of the pixel 92 is given tothe pixel 93.

In this way, the remaining pixels are also labeled.

In step S902, coordinates of a circumscribing rectangle of the adjacentblack pixels are detected. In this step, coordinates are detected of thecircumscribing rectangle enclosing the black pixels having the samelabel.

FIG. 37 is a diagram for schematically showing the operation ofdetecting coordinates of the circumscribing rectangle enclosing theblack pixels having the same label.

As illustrated in FIG. 37, coordinates of a circumscribing rectangle 94enclosing the black pixels having the label “1”, and a circumscribingrectangle 95 enclosing the black pixels having the label “2” aredetected.

In step S903, circumscribed rectangles overlapped with each other, orcircumscribed rectangles in contact with each other, are combined.

FIG. 38 is a diagram for schematically showing the operation ofcombining the circumscribed rectangles.

As illustrated in FIG. 38, circumscribed rectangles 94 and 95, which arein contact with each other, are combined into a new circumscribingrectangle 96.

In step S904, those circumscribed rectangles having sizes not matchingwith the size of the two-dimensional code are ignored, and circumscribedrectangles having reasonable sizes compared with the two-dimensionalcode are extracted.

In this way, a region including black pixels surrounded by white pixelsis detected, and the image processing unit 84 decodes image data in theextracted region.

If the position of the two-dimensional code is already known, it issufficient for the image processing unit 84 to decode the image data ina rectangular region at the position of the two-dimensional code. If theposition of the two-dimensional code is not known from the layout of thepaper document, the image processing unit 84 needs to decode the imagedata in all the detected rectangular regions. If the decoding issuccessful, the image processing unit 84 obtains the paper ID from thedecoding results.

If the decoding is successful, and the image processing unit 84 obtainsthe paper ID, the image processing unit 84 transmits a type of device,the paper ID, and the written information to the element K 131. Forexample, these data can be transmitted in XML form (extensible MarkupLanguage).

The scanner used in the above description may also be the image pickuppart of a digital cameral or an Over Head Cameral (OHC), in addition toa flat bed scanner.

Below, a description is made of the element E 125 for decoding the paperID.

First, a bar-code reading device for reading the two-dimensional code isdescribed as an embodiment of the element E 125.

FIG. 39 is a block diagram showing a configuration of a bar-code readingdevice as an embodiment of the element E 125.

As illustrated in FIG. 39, the bar-code reading device includes a vertexcandidate detection unit 11, a code frame detection unit 12, aprojecting transformation coefficient calculation unit 13, a datasampling unit 14, an error correction unit 15, and a format conversionunit 16.

The bar-code reading device shown in FIG. 39, as the two-dimensionalcode reader 62 in FIG. 29, is above the two-dimensional code and aimedobliquely downward reads the two-dimensional code.

The vertex candidate detection unit 11 detects candidates of thevertices of the two-dimensional code.

The code frame detection unit 12 detects a code frame of thetwo-dimensional code based on the vertex candidates of thetwo-dimensional code, as described below.

The projecting transformation coefficient calculation unit 13 calculatesthe projecting transformation coefficients, which are used in mappingdefined coordinates of centers of cells of the created two-dimensionalcode and coordinates of centers of cells of the two-dimensional coderead by the code frame detection unit 12 from coordinates of thevertices of the code frame read by the code frame detection unit 12, andthe defined coordinates of the vertices of the code frame when creatingthe two-dimensional code.

The data sampling unit 14 uses the projecting transformationcoefficients calculated by the projecting transformation coefficientcalculation unit 13 to sample data of the two-dimensional code.

The error correction unit 15 determines whether errors exist in the datasampled by the data sampling unit 14. If there is no error, or if theerrors are correctable, the error correction unit 15 outputs 56-bit dataof an integral type besides an error correction code to the formatconversion unit 16.

The format conversion unit 16 converts the input data to a characterstring and outputs the character string.

FIG. 40 is a view showing the operation of the vertex candidatedetection unit 11 for detecting the candidates of the vertices of aninput image.

As illustrated in FIG. 40, the vertex candidate detection unit 11 scansan input image, for example, the two-dimensional code, in the directionsindicated by the arrows in order to detect the vertices. Specifically,the vertex candidate detection unit 11 scans the input image from thefour corners of the two-dimensional code along inclined directions todetect black pixels. Here, the obtained black pixels are represented byA, B, C and D.

FIG. 41 is an enlarged view of that in FIG. 40 for showing the operationof determining whether the black pixels A, B, C and D are candidates ofvertices of the input image.

As illustrated in FIG. 41, the vertex candidate detection unit 11 scansthe input image, for example, the two-dimensional code, in thedirections indicated by the arrows originating from the black pixels A,B, C and D, respectively.

For example, considering the black pixel A, the vertex candidatedetection unit 11 scans the two-dimensional code in the right-lowerdirection (45 degrees relative to the vertical line) indicated by thearrow from the black pixel A. For the black pixel B, the vertexcandidate detection unit 11 scans the two-dimensional code in theleft-lower direction (45 degrees relative to the vertical line)indicated by the arrow from the black pixel B. For the black pixel C,the vertex candidate detection unit 11 scans the two-dimensional code inthe right-upper direction (45 degrees relative to the vertical line).For the black pixel D, the vertex candidate detection unit 11 scans thetwo-dimensional code in the left-upper direction (45 degrees relative tothe vertical line).

In this way, the vertex candidate detection unit 11 traces, from pixelsA, or B, or C, or D, a number of pixels equaling 1/√{square root over ()}2 (inversion of the square root of 2) of the number of pixels along aside of a cell, and determines whether all of the scanned pixels areblack pixels. If all of the scanned pixels are black pixels, the vertexcandidate detection unit 11 determines that the black pixels A, B, C andD are candidates of vertices of the two-dimensional code. Then thevertex candidate detection unit 11 transfers control to the code framedetection unit 12.

FIG. 42 is a view showing the operation of the code frame detection unit12 for detecting a code frame of an input image.

As described above, the vertex candidate detection unit 11 traces, fromthe vertex candidates A, or B, or C, or D, a 1/√{square root over ( )}2times number of pixels. Those lines connecting the tracing ends form“black frame determination lines” 76, 77, 78, and 79 for determining ablack frame.

The code frame detection unit 12 determines the black framedetermination lines 76, 77, 78, and 79 as a code frame if the ratio ofthe number of the black pixels over the total number of pixels passed bythe black frame determination lines 76, 77, 78, and 79 is greater thanor equal to 80% for each of the black frame determination lines 76, 77,78, and 79. Further, the code frame detection unit 12 determines thevertex candidates A, or B, or C, or D as vertices, and detects thecoordinates of the vertices A, B, C, and D.

The projecting transformation coefficient calculation unit 13 calculatesthe projecting transformation coefficients, which are used in mappingdefined coordinates of centers of cells of the created two-dimensionalcode and coordinates of centers of cells of the two-dimensional coderead by the code frame detection unit 12 from coordinates of thevertices of the detected code frame, and the defined coordinates of thevertices of the code frame when creating the two-dimensional code.

The method of making this calculation is described below.

Next, a description is given to the data sampling unit 14.

The data sampling unit 14 receives the two-dimensional code, the definedcoordinates of centers of cells of the two-dimensional code whencreating the two-dimensional code, and the projecting transformationcoefficients.

The data sampling unit 14 uses the projecting transformationcoefficients calculated by the projecting transformation coefficientcalculation unit 13 to perform a projecting transformation for thedefined coordinates of centers of cells of the created two-dimensionalcode, and thereby, calculating coordinates of centers of cells of thetwo-dimensional code read by the code frame detection unit 12.

With the obtained coordinates of centers as coordinates of samplingcenters of the image, the data sampling unit 14 specifies a value ofpixels in a 3×3 pixel matrix, whose center is at the sampling center, tobe “1” if the number of black pixels is greater than the number of whitepixels in the matrix, and to be “0” if the number of black pixels is notgreater than the number of white pixels in the matrix, and reads out thevalues. The obtained data having a length of 72 bits are input to theerror correction unit 15, and are arranged therein.

The error correction unit 15 determines whether errors exist in the datasampled by the data sampling unit 14. If there is no error, or if theerrors are correctable, the error correction unit 15 outputs 56-bit dataof an integral type besides an error correction code to the formatconversion unit 16.

The format conversion unit 16 converts the input data to a characterstring and outputs the character string, thus reproducing the originalcharacter string.

The operation of reading the two-dimensional code as described above issummarized in FIG. 43.

FIG. 43 is a flowchart showing the operation of reading thetwo-dimensional code. The operation shown in FIG. 43 may be executed bysoftware.

In step S1001, an image of the two-dimensional code is input. This imageis captured by an image capturing device which is above thetwo-dimensional code and points obliquely downward at thetwo-dimensional code.

In step S1002, candidates of the vertices of the two-dimensional codeare detected.

In this step, as illustrated in FIG. 40, the two-dimensional code isscanned from the four corners of the two-dimensional code along inclineddirections to detect black pixels. The detected black pixels arerepresented by A, B, C and D.

In addition, as illustrated in FIG. 41, the two-dimensional code isscanned from the black pixels A, B, C and D along the directions each atan angle of 45 degrees relative to a vertical line. For example, for theblack pixel A, the scanning is in the right-lower direction (45 degreesrelative to the vertical line), for the black pixel B, the scanning isin the left-lower direction (45 degrees relative to the vertical line),for the black pixel C, the scanning direction is in the right-upperdirection (45 degrees relative to the vertical line), and for the blackpixel D, the scanning is in the left-upper direction (45 degreesrelative to the vertical line).

In these scanning, a number of pixels equaling 1/√{square root over ()}2 of the number of pixels along a side of a cell are traced, and it isdetermined whether all of the scanned pixels are black pixels. If all ofthe scanned pixels are black pixels, the black pixels A, B, C and D aredetermined to be candidates of vertices of the two-dimensional code.

In step S1003, as illustrated in FIG. 42,

The black frame determination lines 76, 77, 78, and 79, which connectthe ends of the above scanning from the vertex candidates A, B, C and Din directions each at an angle of 45 degrees relative to the verticalline, are determined to be a code frame, if the ratio of the number ofthe black pixels over the total number of pixels passed by the blackframe determination lines 76, 77, 78, and 79 is greater than or equal to80% for each of the black frame determination lines 76, 77, 78, and 79.Further, the vertex candidates A, or B, or C, or D are determined to bevertices of the two-dimensional code. At the same time, the coordinatesof the vertices A, B, C, and D are detected.

In step S1004, if the code frame cannot be detected in step S1003, theroutine branches at step S1004 to finish the operation of reading thetwo-dimensional code.

If the code frame is detected in step S1003, the routine proceeds tostep S1005.

In step S1005, the projecting transformation coefficients arecalculated, which are used in mapping the defined coordinates of centersof cells of the created two-dimensional code and the coordinates ofcenters of cells of the two-dimensional code input in step S1001, fromcoordinates of the vertices of the detected code frame detected in stepS1003 and the defined coordinates of the vertices of the code frame whencreating the two-dimensional code.

In step S1006, data sampling is performed. Specifically, after inputtingthe two-dimensional code, the defined coordinates of centers of cells ofthe two-dimensional code when creating the two-dimensional code, and theprojecting transformation coefficients, the projecting transformation isperformed, by using the projecting transformation coefficientscalculated in step S1005, on the defined coordinates of centers of cellsof the created two-dimensional code, thereby, enabling calculation ofcoordinates of centers of cells of the two-dimensional code input instep S1001.

With the obtained coordinates of centers as coordinates of samplingcenters of the image, a value of pixels in a 3×3 pixel matrix, whosecenter is at the sampling center, is specified to be “1” if the numberof black pixels is greater than the number of white pixels in thematrix, and to be “0” if the number of black pixels is not greater thanthe number of white pixels in the matrix, and reads out 72-bit data.

In step S1007, it is determined whether errors exist in the obtained72-bit data. If there is no error, or if the errors are correctable,56-bit integral type data different from the error correction code areoutput.

In step S1008, the 56-bit integral type data is converted to a characterstring.

In step S1009, the character string is output, thus reproducing theoriginal character string.

Next, a description is given to the method of calculating the projectingtransformation coefficients.

The projecting transformation is a well known method in processing threedimensional images. This method transforms a drawing or an object in athree-dimensional space to a drawing or object in a two-dimensionalplane or on a screen. Specifically, the projecting transformationtransforms coordinates of the object in the three dimensional space tothe coordinates of the object on a plane or a screen.

In order to precisely perform the projecting transformation, it isnecessary to know the position relations and optical characteristics ofthe image pick-up system concerned. However, it is difficult to measureor determine the position relation and the optical characteristics of aspecific image pick-up system each time the projecting transformation isperformed for the specific image pick-up system, and this is notdesirable from the point of view of versatility of the transformationrelative to various kinds of devices.

On the other hand, in order to take advantages of an optical systemcapable of obliquely picking up an image of the two-dimensional code, itis not always necessary to carry out the projecting transformationprecisely. For example, if the two-dimensional code is not representedby rectangular cells each having a uniform density, but by trapeziumcells or cells having non-uniform density, it is possible to exhibitadvantages of the optical system capable of obliquely picking up animage of the two-dimensional code.

FIG. 44 is a view for schematically showing an example of reading animage having trapezium cells or having cells of various sizes by anoptical system for capable of obliquely picking up an image.

In the optical system illustrated in FIG. 44, the captured image of arectangular appearance originally is transformed into a trapeziumappearance by the projecting transformation.

In this transformation, the ratio of the long side of the trapezium overthe short side (X1/X3) equals an inversion of the ratio of the distancesfrom the image pick-up element to two ends of the target image, forexample, the two-dimensional code (L3/L1).

In this example, instead of creating a two-dimensional code includingrectangular cells having equivalent sizes, a two-dimensional code iscreated including cells having different shapes and sizes so that theeffect of the projecting transformation is cancelled.

The sizes of the cells are determined for each line of thetwo-dimensional code at a certain magnification. Depending on thedistance to the imaging plane of the image pick-up unit, themagnification of the deformation changes even in the same line of thetwo-dimensional code. For example, the magnifications in the same lineare transformed to the magnification at the middle position of the line,so that the magnifications in the same line are the same. By thistreatment, the two-dimensional codes each having stepwise cellboundaries, as shown in FIG. 45 and FIG. 46, are obtained.

FIG. 45 is a view of an example of a two-dimensional code obtained bytransformation, in which the boundaries of cells are stepwise.

FIG. 46 is a view of another example of a two-dimensional code obtainedby transformation, in which the boundaries of cells are stepwise.

Alternatively, the projecting transformation may be carried out in aprecise way to generate a two-dimensional code as shown in FIG. 47.

FIG. 47 is a view showing of the projecting transformation on atwo-dimensional code.

The details of the two-dimensional codes shown in FIG. 45, FIG. 46, andFIG. 47 are described below referring to the element G 127.

FIG. 48 is diagram showing the projecting transformation fortransforming an object two-dimensional code 67 to an electricallycreated two-dimensional code 68.

FIG. 48 schematically illustrates a two-dimensional code 67, an image ofwhich is to be picked up, and a two-dimensional code 68 electricallycreated; As, Bs, Cs, and Ds represent vertices of the two-dimensionalcode 67, and Ar, Br, Cr, and Dr represent vertices of thetwo-dimensional code 68; xs1, yr2 and others in parentheses representscoordinates; Psk and Prk represent coordinates of centers of cells.

The vertices As, Bs, Cs, and Ds of the two-dimensional code 67, andvertices Ar, Br, Cr, and Dr of the two-dimensional code 68 satisfyequations (5) and (6) in FIG. 49.

FIG. 49 shows equations (5) and (6) for calculating transformationcoefficients.

The equations (5) and (6) express the transformation of coordinates fordefining the two-dimensional code 68 to coordinates for defining thetwo-dimensional code 67, that is, the transformation from Ar, Br, Cr,and Dr to As, Bs, Cs, and Ds.

The equation (5) gives the X coordinate of a vertex, the equation (6)gives the Y coordinate of a vertex, and the subscript i in equations (5)and (6) varies in the range from 1 to 4.

In equations (5) and (6), b1 through b8 are unknown transformationparameters, and these parameters can be determined by substitutingvalues of coordinates of Ar, Br, Cr, and Dr, and As, Bs, Cs, and Ds intoequations (5) and (6), and solving the resulting equation system.

Once b1 through b8 are obtained, the coordinates Psk of sampling centerof the two-dimensional code can be calculated by transforming thecoordinates Prk of center of the cells of the electrically createdtwo-dimensional code.

Next, a description is given to embodiments of the element F 126, whichencodes the paper ID.

FIG. 50 is a block diagram showing a configuration of a two-dimensionalcode creating device.

The element F 126 corresponds to the two-dimensional code creatingdevice, and the element G 127 corresponds to the two-dimensional code.

In FIG. 50, the two-dimensional code creating device shown includes aformat conversion unit 20, an error correction unit 21, and atwo-dimensional code creation unit 23.

The format conversion unit converts the input character string to 56-bitinteger type data able to be handled by a computer.

The error correction unit 21 appends a 16-bit error correction code tothe 56-bit integer type data converted by the format conversion unit 20.

As the error correction code, Reed Solomon code can be used. The errorcorrection method using the Reed Solomon code is powerful in correctingerrors in unit of bytes, and can correct errors shorter than a halflength of the error correction code. For details of Reed Solomon errorcorrection code, please refer to “Code Theory (Lecture on Fundamentalsof Computers (No. 18))”, by Miyakawa et al., and many other books.

In this example, because the length of the error correction code is twobytes, it is possible to correct errors of one byte.

The two-dimensional code creation unit 23 assigns the data and the errorcorrection data to cells of the two-dimensional code to create thetwo-dimensional code.

FIG. 51 is view for schematically showing data arrangement in thetwo-dimensional code creation unit 23. The two-dimensional code creationunit 23 assigns the data and the error correction data to cells of thetwo-dimensional code as shown in FIG. 51.

The numbers in cells shown in FIG. 51 is the numbers assigned torepresent the positions of in the data arrangement.

When assigning the data and the error correction data to cells of thetwo-dimensional code, the 56-bit integer data are assigned to cells 1through 56, and the 16-bit error correction data are assigned to cells57 through 72.

Coordinates of the defined vertices and the coordinates of the cellcenters are used when creating the two-dimensional code, and whenreading the two-dimensional code.

FIG. 52 is flow chart showing the operation of creating thetwo-dimensional code. The operation shown in FIG. 52 can be executed bysoftware.

In step S1101, a character string is input.

In step S1102, the character string is converted to 56-bit integer typedata that is able to be handled by a computer.

In step S1103, a 16-bit error correction code is created and is appendedto the 56-bit integer type data converted in step S1102. As the errorcorrection code, Reed Solomon code is used.

In step S1104, the data and the error correction code are assigned tocells of the two-dimensional code, as shown in FIG. 51 and FIG. 53, tocreate the two-dimensional code.

FIG. 53 is a view for schematically showing data arrangement.

When assigning the data and the error correction code to cells of thetwo-dimensional code, the 56-bit integer data are assigned to cells 1through 56, and the 16-bit error correction code is assigned to cells 57through 72.

Coordinates of the defined vertices and the coordinates of the cellcenters are used when creating the two-dimensional code, and whenreading the two-dimensional code.

Next, a description is given to embodiments of the element G 127, whichcorresponds to the paper ID.

First, the two-dimensional code shown in FIG. 45 is described.

The two-dimensional code shown in FIG. 45 is enclosed by a black frame,and inside the two-dimensional code, cells are arranged in 72 positionsas shown in FIG. 51. A cell is the minimum unit of a zone for recordinginformation, and represents 1-bit data for indicating white or black.Therefore, the two-dimensional code shown in FIG. 45 or FIG. 51expresses information of 72 bits.

When reading the two-dimensional code obliquely downward, thetwo-dimensional code obtained by a bar-code reader exhibits a trapeziumshape, specifically, in the two-dimensional code shown in FIG. 45, areasof the cells near the bottom of the trapezium are large, and thus thebottom of the trapezium is shrunk.

Although the two-dimensional code distorts, each cell in thetwo-dimensional code obtained by the bar-code reader occupies an areaincluding a multiple number of pixels, and the size of the area isroughly constant regardless of the position of the cell in thetwo-dimensional code.

With the trapezium two-dimensional code, in which the cells near thebottom of the trapezium are large, the cells near the top of thetrapezium are small, and with the bar-code reader capable of reading thetwo-dimensional code obliquely downward, it is possible to stably readblack-white data of each cell.

FIG. 46 shows another example of the trapezium two-dimensional code. Inthe trapezium two-dimensional code in FIG. 46, obtained with a bar-codereader that reading the two-dimensional code obliquely downward, thesize of a cell changes depending on the distance from an image pick-upunit to the cell.

In this trapezium two-dimensional code, there are totally six lines, andthere are twelve cells in each line, thus the total number of the cellsis 72. This number is the same as that of the trapezium two-dimensionalcode in FIG. 45, and thus, the two-dimensional code in FIG. 46 canrepresent the same number of bits as the two-dimensional code in FIG.45. That is, the 72-bit data are arranged as illustrated in FIG. 52.

FIG. 47 shows another example of the trapezium two-dimensional code. InFIG. 47, a rectangular two-dimensional code 57 is transformed, by theprojecting transformation, to a trapezium two-dimensional code 58, andthe trapezium two-dimensional code 58 may be used.

In this case, the image of the trapezium two-dimensional code 58, whichis obtained by obliquely downward picking up an image of thetwo-dimensional code, is close to the rectangular two-dimensional code57. The two-dimensional code 58 can represent 72-bit data.

The three two-dimensional codes described above are similar in that allof them are larger in widths than in heights. If the cells in thetwo-dimensional codes are taller, the heights of cells are large, andthereby, the total sizes of the two-dimensional codes become small, andthe two-dimensional code loses the merit of displaying more informationwith smaller size.

In addition, when viewed from the bar-code reader, because the distancefrom the bar-code reader to the two-dimensional code changes more in thevertical direction than in the horizontal direction, it is appropriateto set cells far from the bar-code reader to have a large height-widthratio. Therefore, in FIG. 46 and FIG. 47, the height of the cell changesmore than the width of the cell, and the two-dimensional code read bythe bar-code reader is nearly a square.

Next, a description is given to embodiments of the element H 128, whichassociates the process ID with the external program 111, and manages theprocess ID.

When other elements make a request to the element H 128 of a process IDof an external program (registration request), in a computer installedwith as least the element H 128, the element H 128 receives informationfor uniquely defining the external program, and assigns a unique ID inthe management table in response to the request, and manages informationof the external program in the management table.

FIG. 54 is a view showing an example of a management table.

In FIG. 54, each line, for example, the line 155, is a unit of theexternal program corresponding to the process ID, and each of columns151, 152, 153, 154 includes registered elements of the process IDs.

Listed in column 151 are the assigned process IDs. In column 152, thereis the information for uniquely specifying an external program. Forexample, a file path of the external program is stored in the column 152as the first item, and whereby the external program is uniquelyspecified. Stored in the second item of the column 152 is a path nameindicating a name of a folder for storing jobs. Here, it is assumed thatthe external program processes files in one job folder, and thecorrespondence relation between them are uniquely defined, thereby theprocess IDs in the column 151 are associated with external programs.

The example shown by the second line in FIG. 54 says that a program forprocessing a fixed-format document is registered to have a process ID of0001; the program receives, as a job in a folder“c:¥JobFolder¥WordPrint”, written information from a paper documentobtained by printing an electronic document *.doc.

In the third item of the column 152, a program in other computer isspecified as a Web service by using its URL. The URL is connected to theprocess ID.

In the column 153, there is stored attribute information of electronicdocuments, by printing the electronic documents, paper documents areobtained, and the external program deals with the written informationfrom paper documents. In the column 154, there are stored descriptionsof the external programs. There may be more columns like the columns153, and 154 for storing attribute information related to the externalprograms.

When a process ID is specified and management information of an externalprogram is requested (referred to as “reference request”), the element H128 searches for an appropriate process ID from the process IDs in thecolumn 151 in FIG. 54. The element H 128 transmits, to the requester, anitem of the registered element in the column 152 corresponding to theobtained process ID, which uniquely specifies a folder path name. Atthis time, the element H 128 may also transmit the attribute informationin the column 153 and the subsequent columns simultaneously.

When the element H 128 is a program, the contents of the managementtable in FIG. 54 may be stored in the memory used by the program inexecution. Alternatively, the contents of the management table may bestored as a file in a file system at a certain timing. Further, thecontents of the management table may be stored in a database as digitalinformation. In this case, one process ID and the correspondingattribute information form one record, and the process ID may be used asa key for storing or retrieving information related to the externalprograms, thereby enabling association of the process IDs with theexternal programs and enabling management of the information.

In this way, information of association of the process IDs with theexternal programs controlled by the element H 128 in the managementtable is stored not only in the memory used by executing programs, butalso in a non-volatile storage device such as a hard disk as a file.Therefore, even when the power of the element H 128 is turned off, theassociation information may be evoked again for later use.

In addition, because the process ID and the name of the directory inwhich the external programs corresponding to the process ID works arestored and controlled by the element H 128, even if the name of theprogram or the program itself changes, the element H 128 can deal withthis without changing the management information in the element H 128.

Furthermore, the element H 128 may be provided in a device independentfrom the device provided with the other elements, and the device havingthe element H 128 and the device having the other elements may beconnected with each other by a communication network so that the devicehaving the other elements can access the device having the element H128. With this configuration, the element H 128 functions as RPC (RemoteProcedure Call) or a Web service, and accepts registration requests orreference requests from other elements via the communication network.

In this case, information of the correspondence relation between theprocess IDs and the external programs held in the management table maybe stored in the non-volatile storage device as a database able to beaccessed via a communication network. Thereby, it is possible to providemore than one elements H 128 in the system, which registers or refers tothe correspondence relation information in the same way.

Because even external programs on other devices can be specified via thecommunication network, it is possible to install the external programsto be executed on host devices respectively matching with theoperational conditions of the external programs.

Because the device in the system having the element H 128 only isconnected with the device having the other elements via thecommunication network, the device having the other elements canassociate the process IDs and the external programs via thecommunication network, and manage the information. The devices havingthe other elements distributed on the communication network can alsoregister or refer to the correspondence relation information in the sameway in the system.

Next, a description is given to embodiments of the element I 129, whichedits and manages a processing method and form layout information.

The element I 129 reads a paper document, converts recognized writteninformation based on a layout defined by the paper document andprocessing information, and stores the conversion results in aninformation table described below.

FIG. 55 is a view of an example of a paper document (that is, a form).Shown in FIG. 55 is a filled-out registration form for entrance to orresignation from a club. This registration form may be the registrationform 47 shown in FIG. 11B.

The registration form in FIG. 55 includes an input field A 170, an inputfield B 171, an input field C 172, an input field D 173, and an inputfield E 174. The registration form also includes a name label 175, amembership number label, a request label, and a comment label.

FIG. 56 is a view of an information table.

The information table stores registration information and processinginformation. The registration information includes IDs, names, andmembership numbers. The processing information includes names,membership numbers, processing, and comments.

In other words, the information written down on the registration form isstored in the information table.

The information provided in the field A 170, field B 171, and field E174 of the registration form is stored in columns of “name”, “membershipnumber”, and “comment”, respectively, of the information table shown inFIG. 56. The information provided in the field C 172 and field D 173 ofthe registration form is stored in items of “deletion” or “registration”depending on the check mark placed in the registration form.

Next, a description is given to a form creation program. In thefollowing description, a file that defines a processing method and formlayout information is called “form definition entity”, and software thatedits the form definition entity is called a form creation program.

Below, the system shown in FIG. 1 is used as the apparatus relevant tothe form creation program.

The form creation program is executed on the computer 101 that preparespaper forms, and a database program is running on the data server 106.The database program controls input and output of the form definitionentity generated by the form creation program, and carries out input andoutput of the information table used in form processing.

The form creation program defines a method of processing a form andlayout information of a form, as illustrated in FIG. 55, as a formstructure entity, and creates contents of the form definition entity.

FIG. 57 is a view showing a form structure entity.

The form structure entity shown in FIG. 57 divides the input field A 170into sessions by meaning of properties, for example, it uses an elementlabel <Layout> to manage the layout information of the form, an elementlabel <Input> to manage recognition information, and an element label<Registration> to manage the storage address, and the contents of theselabels are described by properties of XML.

FIG. 58 is view for schematically showing an interface of the formcreation program.

As illustrated in FIG. 58, the form creation program has a GUI(Graphical User Interface). This GUI includes a layout area 155, and atool box 156. The tool box 156 includes a label tool 151, a text box152, a check box 153, and a bar code 154.

A person who desires to create a form uses the tool box 156 to arrangelabels and text boxes in the layout area 155, thereby creating a form.

The label tool 151 creates labels in the layout area 155, the text box152 creates text boxes in the layout area 155, the check box 153 createscheck boxes in the layout area 155, and the bar code 154 creates a barcode in the layout area 155.

In this way, contents corresponding to the components of the tool box156 are arranged in the layout area 155, and their properties are set onthe screen. Here, by “content”, it means those objects arranged by thelabel tool 151, the text box 152, the check box 153, and the bar code154.

FIG. 59 is view for schematically showing creation of a field forinputting a name by using the text box 152.

FIG. 60 is a view of a table including groups of properties of contents.These properties define the actual contents of the form and the methodsof processing them.

FIG. 60 shows an example of properties corresponding to the name label175 and the input field A 170 shown in FIG. 55. The properties areroughly classified into layout information 176 (from “content beginning”to “border”), recognition information 177 (from “input” to “recognitionresult”), and storage address information 178 (from “registered server”to “registered record”).

The layout information 176 controls positions and sizes of contents, andstate of characters and frame lines displayed there. The recognitioninformation 177, when processing a form, controls the method forconverting hand written information written in the input fields to data,and includes information used for character recognition.

The storage address information 178 controls storage address ofconversion results by the recognition information 177 used whenprocessing a form.

In FIG. 60, for example, for the content having an ID=0001, since its“type” is “Label”, this is a label. Since “character” of this label is“Name”, and “border” is “No”, this label displays the character “Name”without a border. Since “Input” of this label is “disabled”, no specialprocessing is performed when the label is filled out. Since “Conversion”of this label is “No”, no conversion is performed on the inputcharacters. Since the storage address information 178 is empty, thereare no data to be saved.

For the content having an ID=0002, its “type” is “Textbox”, so this is atextbox. Since “character” of this textbox is empty, there is nocharacter displayed. Since “border” of the textbox is “enclosing”, thistextbox displays an enclosing frame line. Since “Input” of this textboxis “enabled”, some kind of special processing is performed forconverting the characters displayed by “recognition information 1” to“recognition information 3”, that is, “Japanese”, “handwriting”, and“Name”. The recognized results are stored at places described in thestorage address information 178.

As described above, by providing the form structure entity with suchproperties, the form layout information and processing method aredefined.

Next, a description is given to embodiments of the element J 130, whichdecomposes the hand-written information into data based on form layoutinformation and the processing method, and stores the decomposedhand-written data.

FIG. 61 is a view for schematically showing decomposition ofhand-written information into data.

When a scanner is used as the element D 124, with the registration formfor entrance to or resignation from a club in FIG. 61 as an example, theelement J 130 compares the captured image 180 of the filled-outregistration form and the image 181 of the unfilled-out registrationform, and thereby obtains an image of the hand-written portion, whichcorresponds to a difference between the image 180 and the image 181. Theelement J 130 then decomposes the hand-written portion into name,membership and other contents and stores these contents.

When a coordinate input device is used as the element D 124, coordinatesof the hand-written portion, that is, the hand-written information, canbe obtained directly, and the element J 130 decomposes the hand-writtenportion into name, membership and other contents and stores thesecontents.

When processing an unfixed-format document, because the concept “format”is not used in this case, decomposition is not performed, but thedocument is stored as a whole.

When a scanner is used as the element D 124, as described above, theimage of a filled-out form and the image of the unfilled-out form arecompared to obtain a difference between the images to extract thehand-written portion.

When a coordinate input device is used as the element D 124, thehand-written information can be obtained directly.

FIG. 62 is a flowchart showing the operation of decomposing hand-writteninformation written down on a form into data and storing the decomposedhand-written data.

In step S1201, the form is recognized.

In step S1202, an image of the unfilled-out form is generated. In thisstep, the image of the unfilled-out form is generated from the formdefinition entity called in the step 1201.

In step S1203, positions of the images of the unfilled-out form and thefilled-out form are aligned. In this step, a timing mark is used tomatch the filled-out form with the unfilled-out form. The position andshape of the timing mark are obtained from the form definition entity.

If the timing mark does not exist, this step can be omitted.

In step S1204, an image of the hand-written portion is generated. Inthis step, the image of the filled-out form and the image of theunfilled-out form are compared, and the difference between the twoimages is obtained to extract the hand-written portion.

Step S1205 through step S1208 are performed for each content unit shownby the form definition entity.

In step S1205, partial images corresponding to the contents aregenerated. This step is carried out according to the properties of thecontent (layout information). For example, images of the contents“Yamata Tarou”, “56-381” in FIG. 61 are generated.

In step S1206, character recognition is performed for each partialimage. This step is carried out according to the properties of thecontents (recognition information).

In step S1207, after character recognition, each partial image isconverted into characters, and the obtained characters are stored.

In step S1208, it is determined whether all contents are processed. Ifun-processed contents exist, the routine goes to step S1205 to repeatthe above steps. If all contents have been processed, the routine isfinished.

Next, a description is given to embodiments of the element K 131, whichconverts the paper ID to the document ID and the process ID.

FIG. 63 is block diagram of the relation between the element K 131 andother elements, showing the operation of the element K 131 forconverting the paper ID to the document ID and the process ID.

As illustrated in FIG. 63, with the element K 131 at the center, therectangles represent the elements A through L, the blowouts representdata or contents of requests, the arrows indicate the directions of theflow of the data or the requests, and the step numbers indicate thesequence.

In step S1301, the element K 131 acquires the paper ID and the writteninformation from the D-PAD. Alternatively, the element K 131 uses thescanner or the scanner of a MFP to read a paper document that is handwritten to acquire an image of the paper ID.

In step S1302, the element K 131 sends the paper ID to the element B122, and requests management information related to the paper document.

In step S1303, the element K 131 obtains the process ID or theinformation for uniquely specifying the electronic documentcorresponding to the paper document. The step S1303 corresponds to thestep of “associating the process ID with a program” as defined in theclaims.

In step S1304, the element K 131 sends the process ID to the element H128, and requests management information on the program for processingthe paper document.

In step S1305, the element K 131 obtains the information for uniquelyspecifying a folder path name which is registered by using an identifiercorresponding to the element H 128.

In step S1306, the element K 131 stores all information into a file, andstores the file in the folder specified by the folder path name obtainedin step S1306. Here, the element K 131 stores in the file allinformation obtained by using an image input device, such as the D-PAD,or the scanner or the scanner of a MFP, through the element B 122 andthe element H 128.

In step S1307, the routine proceeds to the next operation in which theelement K 131 transmits jobs to the element J, which decomposes thehand-written information into data based on form layout information andprocessing methods and stores the decomposed hand-written data, or to aunfixed-format document processing program.

While the present invention is described with reference to specificembodiments chosen for purpose of illustration, it should be apparentthat the invention is not limited to these embodiments, but numerousmodifications could be made thereto by those skilled in the art withoutdeparting from the basic concept and scope of the invention.

This patent application is based on Japanese Priority Patent ApplicationNo. 2003-197851 filed on Jul. 16, 2003, Japanese Priority PatentApplication No. 2003-336055 filed on Sep. 26, 2003, and JapanesePriority Patent Application No. 2004-156241 filed on May 26, 2004, theentire contents of which are hereby incorporated by reference.

1. A document processing system for processing a hand-written firstdocument on a sheet, said document processing system comprising: anencoding unit configured to encode a sheet ID for identifying the firstdocument to generate a coded sheet ID; a decoding unit configured todecode the coded sheet ID; a document-sheet ID association unitconfigured to associate the sheet ID with a document ID assigned to acomputerized second document; a printing unit configured to acquire thesheet ID and print the coded sheet ID on the first document; a sheet IDmanagement unit configured to manage the sheet ID; an informationacquisition unit configured to acquire the sheet ID decoded by thedecoding unit, and hand-written data from the first document on whichthe coded sheet ID is printed; and a process-sheet ID association unitconfigured to associate the sheet ID with a process ID of a process forprocessing the hand-written data acquired by the information acquisitionunit.
 2. The document processing system as claimed in claim 1, whereinthe printing unit comprises a printing service unit that prints thefirst document based on the second document, wherein the printingservice unit includes: a first unit that acquires the sheet ID assignedto the first document; a second unit that sends the acquired sheet ID tothe encoding unit to generate the coded sheet ID; and a third unit thatsuperposes the coded sheet ID on the second document.
 3. The documentprocessing system as claimed in claim 1, wherein the printing unitincludes: a first unit that acquires the sheet ID assigned to the firstdocument; a second unit that sends the acquired sheet ID to the encodingunit to generate the coded sheet ID; a third unit that superposes thecoded sheet ID on the second document; and a printing service unit thatoutputs the second document superposed with the coded sheet ID forprinting the first document.
 4. The document processing system asclaimed in claim 1, wherein the printing unit comprises a printingservice unit that prints the first document based on a printing image,wherein the printing service unit includes: a document conversion unitthat converts the second document to the printing image; a first unitthat acquires the sheet ID assigned to the first document; a second unitthat sends the acquired sheet ID to the encoding unit to generate thecoded sheet ID; and a third unit that superposes the coded sheet ID onthe printing image.
 5. The document processing system as claimed inclaim 1, wherein the printing unit comprises: a document conversion unitthat converts the second document to a printing image or a pagedescription language sequence; and a printing service unit that printsthe first document based on the printing image or a page descriptionlanguage sequence, wherein the printing service unit includes a firstunit that acquires the sheet ID assigned to the first document; a secondunit that sends the acquired sheet ID to the encoding unit to generatethe coded sheet ID; and a third unit that superposes the coded sheet IDon the printing image.
 6. The document processing system as claimed inclaim 1, wherein the sheet ID management unit stores document-sheet IDassociation information in a non-volatile storage unit, saiddocument-sheet ID association information being used for associating thesecond document with the sheet ID.
 7. The document processing system asclaimed in claim 6, wherein the document-sheet ID associationinformation is accessible from a communication network.
 8. The documentprocessing system as claimed in claim 1, wherein the sheet ID managementunit associates the second document with the sheet ID in response to arequest transmitted from a communication network.
 9. The documentprocessing system as claimed in claim 8, wherein the sheet ID managementunit associates a document title of the second document with the sheetID.
 10. The document processing system as claimed in claim 8, whereinthe sheet ID management unit stores the second document associated withthe sheet ID.
 11. The document processing system as claimed in claim 1,wherein the information acquisition unit comprises: an image pick-upunit that picks up an image of an object including the first document; adriving unit that drives the image pick-up unit relative to the firstdocument; a control unit that controls the driving unit to divide thefirst document into a plurality of regions for image picking-up; and acombination unit that combines images of the divided regions of thefirst document into one image.
 12. The document processing system asclaimed in claim 11, wherein the information acquisition unit comprises:a sheet ID image pick-up unit that picks up an image of the coded sheetID.
 13. The document processing system as claimed in claim 1, whereinthe information acquisition unit comprises: an image pick-up unit thatpicks up an image of an object including the first document; a drivingunit that drives the image pick-up unit relative to the first document;a detection unit that detects a position of the coded sheet ID on theobject; and a control unit that determines a position of the imagepick-up unit so that the image pick-up unit is able to picks up an imageof the coded sheet ID, and determines a magnification to be applied forpicking up an image of the coded sheet ID; wherein the image obtained bythe image pick-up unit is associated with the sheet ID obtained from theimage of the coded sheet ID.
 14. The document processing system asclaimed in claim 1, wherein the information acquisition unit comprises:an image pick-up unit that picks up an image of an object including thefirst document; and a sheet ID display unit provided in the object fordisplaying the coded sheet ID, said sheet ID display unit being storableand including the coded sheet ID.
 15. The document processing system asclaimed in claim 1, wherein the information acquisition unit comprises:an image pick-up unit that picks up an image of an object including thefirst document; and an optical projecting unit that optically projectsthe coded sheet ID so as to allow the image pick-up unit to pick up animage of the coded sheet ID.
 16. The document processing system asclaimed in claim 1, wherein the information acquisition unit comprises:an image pick-up unit that picks up an image of an object including thefirst document; and a display unit provided in the object for displayingan image of the coded sheet ID taken by the image pick-up unit.
 17. Adocument processing system for processing a hand-written first documenton a sheet, said document processing system comprising: an encoding unitconfigured to encode a sheet ID for identifying the first document togenerate a coded sheet ID; a decoding unit configured to decode thecoded sheet ID; a document-sheet ID association unit configured toassociate the sheet ID with a document ID assigned to a computerizedsecond document; a printing unit configured to acquire the sheet ID andprint the coded sheet ID on the first document; a sheet ID managementunit configured to manage the sheet ID; an information acquisition unitconfigured to acquire the sheet ID decoded by the decoding unit andhand-written data from the first document on which the coded sheet ID isprinted; a process-sheet ID association unit configured to associate thesheet ID with a process ID of a process for processing the hand-writtendata acquired by the information acquisition unit; and an ID conversionunit configured to convert the sheet ID to the document ID and theprocess ID.
 18. The document processing system as claimed in claim 17,wherein the information acquisition unit comprises: an image pick-upunit that picks up an image of an object including the first document;and a detection unit that detects a position of the coded sheet ID onthe object; wherein the sheet ID obtained from an image of the codedsheet ID taken by the image pick-up unit is associated with the documentID and the process ID by the ID conversion unit.
 19. A documentprocessing system for processing a hand-written first document on asheet, said document processing system comprising: an encoding unitconfigured to encode a sheet ID for identifying the first document togenerate a coded sheet ID; a decoding unit configured to decode thecoded sheet ID; a printing unit configured to acquire the sheet ID andprint the coded sheet ID on the first document; a document-sheet IDassociation unit configured to associate the sheet ID with a document IDassigned to a second document, said second document being a computerizeddocument having a predetermined format; a sheet ID management unitconfigured to manage the sheet ID; an information acquisition unitconfigured to acquire the sheet ID decoded by the decoding unit and thehand-written data from the first document on which the coded sheet ID isprinted; a process-sheet ID association unit configured to associate thesheet ID with a process ID of a process for processing the hand-writtendata acquired by the information acquisition unit; a process IDmanagement unit configured to manage the process ID; and a decompositionstorage unit configured to decompose the hand-written data acquired bythe information acquisition unit based on layout information of thepredetermined format and the process ID, and store the decomposedhand-written data.
 20. The document processing system as claimed inclaim 19, wherein the process ID management unit manages program-processID association information that associates the process ID with a programfor executing the process for processing the hand-written data.
 21. Thedocument processing system as claimed in claim 20, wherein the processID management unit stores the program-process ID association informationin a non-volatile storage unit.
 22. The document processing system asclaimed in claim 20, wherein the program-process ID associationinformation is stored in a database accessible from a communicationnetwork.
 23. The document processing system as claimed in claim 20,wherein the program-process ID association information includesinformation of a position of storing data to be processed by theprogram.
 24. The document processing system as claimed in claim 20,wherein the program-process ID association information includesinformation of a position of a service to be provided by executing theprogram.
 25. The document processing system as claimed in claim 20,wherein the process ID management unit associates the process ID withthe program in response to a request transmitted from a communicationnetwork.
 26. A document processing method for processing a hand-writtenfirst document on a sheet, said document processing method comprisingthe steps of: associating a sheet ID with a process ID, said sheet IDbeing used for identifying the first document, said process ID beingused for indicating a process for processing hand-written data on thefirst document; associating the sheet ID with a document ID assigned toa computerized second document; associating the process ID with aprogram for executing the process for processing the hand-written data;assigning the sheet ID to the first document; encoding the sheet ID togenerate a coded sheet ID; printing the first document with the codedsheet ID thereon; acquiring the coded sheet ID and the hand-written datafrom the first document, the coded sheet ID being printed on the firstdocument; and converting the sheet ID to the document ID and the processID.
 27. A document processing method for processing a hand-written firstdocument on a sheet, said document processing method comprising thesteps of: associating a sheet ID with a process ID, said sheet ID beingused for identifying the first document, said process ID being used forindicating a process for processing hand-written data on the firstdocument; associating the sheet ID with a document ID assigned to acomputerized second document having a predetermined format; assigningthe sheet ID to the first document; encoding the sheet ID to generate acoded sheet ID; acquiring the coded sheet ID and the hand-written datafrom the first document, the coded sheet ID being printed on the firstdocument; and decomposing the hand-written data based on layoutinformation of the predetermined format and the process ID and storingthe decomposed hand-written data.
 28. A computer-readable storage mediumstoring a computer program for processing a hand-written first documenton the sheet, the computer program, when executed by a processor,executing the steps of: associating a sheet ID with a process ID, saidsheet ID being used for identifying the first document, said process IDbeing used for indicating a process for processing hand-written data onthe first document; associating the sheet ID with a document ID assignedto a computerized second document; associating the process ID with aprogram for executing the process for processing the hand-written data;assigning the sheet ID to the first document; encoding the sheet ID togenerate a coded sheet ID; printing the first document with the codedsheet ID thereon; acquiring the coded sheet ID and the hand-written datafrom the first document, the coded sheet ID being printed on the firstdocument; and converting the sheet ID to the document ID and the processID.
 29. A computer-readable storage medium storing a computer programfor processing a hand-written first document on the sheet, the computerprogram, when executed by a processor, executing the steps of:associating a sheet ID with a process ID, said sheet ID being used foridentifying the first document, said process ID being used forindicating a process for processing hand-written data on the firstdocument; associating the sheet ID with a document ID assigned to acomputerized second document having a predetermined format; assigningthe sheet ID to the first document; encoding the sheet ID to generate acoded sheet ID; acquiring the coded sheet ID and the hand-written datafrom the first document, the coded sheet ID being printed on the firstdocument; and decomposing the hand-written data based on layoutinformation of the predetermined format and the process ID and storingthe decomposed hand-written data.